Agent orchestration is the control layer that sits above a multi-agent workflow and decides which agent runs next, with what input, under what retry policy, and what happens when one of them fails. Without orchestration, several agents are just several scripts; with it, they are one production system.
What orchestration owns.
- Routing. Given the current state, decide which agent handles the next step. Static (planner agent picks) or dynamic (rule engine + state).
- Ordering and dependencies. Which steps run in parallel, which must wait. The orchestrator holds the DAG; the agents don't need to know about each other.
- Retry policy. What to do when an agent fails the rubric, returns malformed output, or exceeds its token budget. Retry on the same agent, escalate to a larger model, route to a human checkpoint, fail closed.
- State and durability. Long-running workflows (minutes to hours) survive process restarts. The orchestrator persists state to durable storage and replays cleanly.
- Observability. Every routing decision is part of the trace. The operator can see why the system did what it did, end to end.
Common patterns.
Three orchestration shapes ship most often in production. DAG (static graph, deterministic routing). Planner-Executor (a planner agent decides at each turn). Hybrid (a static backbone with planner-driven branches at key decision points). The DAG is easiest to evaluate; the planner is most flexible; the hybrid is what most production systems converge on.
Orchestration vs. “just let the LLM decide”.
A common temptation is to ask the LLM to pick the next agent at every turn. It works in demos and fails in production: routing accuracy drops, retries multiply, state becomes invisible. Deterministic routing for the shape, LLM judgement for the leaf decisions, is the production answer.
Frequently asked.
- What is agent orchestration?
- Agent orchestration is the control layer that routes work between agents in a multi-agent workflow, enforces ordering, manages retries, and surfaces the state of the run. Without it, several agents are several scripts; with it, they are one production system. The orchestrator typically holds the DAG, the retry policy, and the durable state.
- Should the LLM decide which agent runs next?
- Sometimes, but not at every turn. Pure LLM-driven routing works in demos and drifts in production. The production pattern is deterministic routing for the shape of the workflow plus LLM judgement at specific leaf decisions where flexibility is needed. The trade is evaluability for flexibility; most workflows want more evaluability than they think.
- What tools does Morvion use for orchestration?
- Workflow engines like Temporal or Inngest for durable state, LangGraph or custom TypeScript DAGs for the routing shape, and the in-house observability layer for trace and replay. The choice depends on durability needs and on whether the workflow runs in seconds or hours.
- How does orchestration relate to the eval harness?
- The eval harness scores per-agent outputs against per-agent rubrics. The orchestrator decides what happens when an output fails the rubric: retry, escalate, route to human, fail closed. Without an eval harness, the orchestrator has no signal to act on; without an orchestrator, the eval harness has no enforcement mechanism. They ship together.