The Orchestra

There’s a moment in building an orchestration layer that I didn’t expect.

You sit down to build an AI system. You’ve spent weeks thinking about the model, which one, how it’s prompted, how it retrieves, how it reasons. That’s the part that feels like the work. And then you open the orchestrator, the piece that actually runs the show, and you start writing. Conditionals. State transitions. Retry logic. A branch for when retrieval returns nothing. A branch for when it returns too much. A rule for when to call a tool instead of the model at all.

Somewhere in that file, you notice the call to the language model is one line.

One step among many. The rest of the file is retries, routing, state transitions, business rules, fallbacks, approvals, context management.

You pause.

When did the model stop being the system?

I didn’t arrive at that question as an abstraction. I arrived at it while working on a GPT-RAG deployment, where the pattern is stark enough to be unmissable: the orchestration layer is where the business logic lives. Not the retriever. Not the model. The orchestrator decides when to retrieve, when to answer directly, when to escalate, when to give up and say so. It holds the rules that make the system trustworthy in production rather than merely impressive in a demo.

Which means something uncomfortable, if you’ve spent years assuming the model is the star: you can swap in a better language model and the system barely improves. You can get the orchestration wrong and no model, however capable, saves you. The component doing the least conventionally “intelligent” work is the one the whole system’s correctness depends on.

That’s the inversion worth sitting with. We built decades of AI research around making a better soloist, a model that reasons further, remembers more, hallucinates less. And that work matters. But a system that works in the world was never going to be one exceptional model answering in isolation. It was always going to be many components, each doing something narrower than “intelligence,” arranged so that the right thing happens at the right moment. Retrieval that knows when to defer. A tool call that knows when it’s needed. A fallback that knows it’s a fallback.

Call that arrangement what it is: an orchestra. Not as a metaphor to be extended, I’m not interested in mapping conductors to routers and violins to agents. Call it that because the word already means what it should mean: many specialized parts, coordinated toward an outcome none of them could produce alone. Orchestration, as a term of art, got there before AI did. We’re just discovering how literal it was always going to be.

First, we engineered algorithms.

Then we engineered models.

Now we’re engineering interactions, between models, tools, data, rules, and people.

That’s a different discipline than the one most of us trained in, and most of us are learning it on the job, mid-deployment, the way I did.

Once you see it in GPT-RAG, you start seeing it everywhere the field is heading. MCP doesn’t make any individual agent smarter, it gives previously incompatible components a shared way of calling each other, the way a common notation lets musicians who’ve never met play the same piece. Agent-to-agent protocols aren’t about upgrading any single agent’s reasoning; they’re about letting separate reasoning processes hand work to each other without losing context in the handoff. Multi-agent systems don’t outperform single strong models because each agent is smarter alone, they outperform because the system encodes when each one should act, and, just as importantly, when it shouldn’t.

None of these are the story. They’re all instances of the same underlying shift: the engineering effort, and the value, is migrating from the model to the layer that decides how models, tools, and data get used.

It isn’t that model quality stops mattering, frontier models still differ meaningfully in reasoning, cost, latency, what they can be trusted with. But as high-quality models become increasingly accessible, the advantage shifts to how they’re orchestrated inside a real system. That layer encodes something no model ships with: your business logic, your failure modes, your definition of when the system should trust itself and when it shouldn’t. That’s not a smaller thing to build. It might be the harder thing, and the more durable one.

I’ve thought before about how the internet solved communication but not collective action, how connecting everyone didn’t, on its own, produce coordination. The same gap is opening up in AI. Building a smarter model isn’t enough, for the same reason connecting more people wasn’t enough. Intelligence, like connection, doesn’t produce outcomes by itself. Outcomes emerge when intelligence is coordinated, with memory, tools, data, rules, other systems, other people.

The next frontier isn’t intelligence.

It’s coordination.

So, I go back to the moment I started with, sitting down to build what I thought was an AI system, and finding a decision architecture instead, with a model as one dependency among several.

The intelligence was never going to be the hard part for long.

Knowing when to use it is.

That’s the orchestra.

Further reading

The architectural observations in this article were inspired by building enterprise AI solutions around Microsoft’s GPT-RAG reference architecture. If you’re interested in the implementation details, the official repository is an excellent companion resource.