I want to share a story that challenged my most basic ideas about software engineering.

Months ago, as part of my first new AI project, I designed a DSL for expressing software designs. I found a pretty friendly way to describe a pseudo-graph of operations and data. You send it to my server, with a target node.

My server builds outside-in to reach the target node. At each node, it prompts a model to implement a suitable function, with the code for already-implemented consumers (the node’s parents in the graph) and a plain English description (from the DSL) supplied as system instructions. Engineers who work outside-in and consumer-first might already know why this is a useful approach for AI.

You can design and run complex things neatly and correctly – take-home programming exercise level stuff. Though I’m working on something else now, I still love that project. I’m proud of two decisions in particular.

First decision: Pass the graph you want to execute and a target node in at runtime and get the result synchronously – not the software, but the result of running it, providing inputs as the outermost edges of your graph. The server doesn’t build software for you, it computes for you, with software that it builds incidentally to get your result. This means there is no separation between software development and operation – only latency and caching.

It is a shock to work with something that does lots of your job at runtime. Our code is usually so precious to us. Think of the care we take in its production, testing, and deployment. What a strange feeling, to suddenly think of it as so trivial that it might not even be worth caching!

Second decision: Let the client patch their graph – just keep adding to it in separate HTTP calls, specifying whatever parts are needed, referencing parts specified earlier, all while targeting a node in each call to get results.

I used it super-iteratively, designing software with a tight feedback loop between my requests and the resulting responses and software until I had a system I liked. At that point, I could delete the graph and any code, because my requests essentially were my software. Everything after design was incidental – generable at runtime. I could use my honed set of graph-segment-and-target-node patch requests like endpoints to operate the system, even if the system didn’t exist yet. It was a Keanu “woah” moment for me.

I thought deeply about this paradigm we are swimming in – where software must be developed and deployed well in advance of being executed. It felt like seeing water.

We could have a dynamic runtime where models run code ad-hoc to achieve their goals. Code can be used, rather than developed, deployed, and executed, collapsing the space between development and runtime as my project did. There’s no technical obstacle, really. We just need time, compute, and creativity.

That glimpse of the future was really impactful for me, and I took the lessons in my following work.