Most teams that reach for multi-agent architecture do it for the wrong reason. Not because a single agent hit a hard limit. Because multi-agent sounds more serious. More capable. More production-ready.
It usually is not. A 2025 Google DeepMind study found that unstructured multi-agent networks — what the researchers call a "bag of agents" — produce up to 17.2× error amplification compared to a well-structured single-agent baseline. More agents, without the right coordination architecture, does not reduce errors. It compounds them.
Multi-agent design adds coordination overhead, multiplies failure modes, scatters permissions across the system, and makes the output harder to review. You pay that cost whether you needed to split the work or not.
The architecture ladder in this article gives you a way to answer the right question first: Does this task actually need more than one agent? Most of the time, the answer is no — until it isn't.
What an Agent Actually Is
Before the ladder makes sense, the term needs a tight definition. For the purposes of this article:
An agent is a language model with access to tools, running a work loop until a goal is reached or a limit is hit.
That is it. The tools might be code execution, file reads, web search, API calls, or a terminal. The loop might be a simple chain or a complex plan-execute-reflect cycle. But the core unit is one model, one tool set, one task.
A single agent is already surprisingly capable. It can read a codebase, identify a bug, propose a fix, run tests, and produce a diff — all in one continuous loop. The question is not whether one agent is capable enough. The question is whether the task has properties that one agent structurally cannot handle alone.
There are exactly three such properties.
The Three Hard Limits
These are not about difficulty. A hard task is still a one-agent task. These are structural constraints — properties of the work itself that no amount of prompting or model quality resolves.
Limit 1: Context Overflow
A language model has a finite context window. For most production tasks, a single agent fits comfortably within it. But some tasks have legitimate scale:
- Analyzing an entire large monorepo for security vulnerabilities
- Comparing implementations across dozens of microservices
- Processing and synthesizing a long corpus of documents
When the input volume reliably exceeds the context window, a single agent cannot hold the full picture. That is not a model limitation to optimize around — it is a structural constraint of the task.
Think of a financial audit spanning a company with offices in ten countries. No single auditor can hold all the ledgers, all the local regulations, and all the transaction histories simultaneously. The natural solution is to assign one auditor per region, have each produce a findings report, and have a lead auditor synthesize the whole picture. The lead never reviews every transaction — they review the summaries. That is exactly what divide and conquer means for agents: each worker holds a scoped slice it can fully process, and the orchestrator holds only the synthesis.
Test: Does the task require simultaneous awareness of more content than fits in the context window? If no, context overflow is not your limit.
Limit 2: Parallelism
Some tasks have independent subtasks that would benefit from running concurrently rather than sequentially. A single agent is serial — it works through one step at a time.
Examples:
- Running security analysis, dependency audit, and test coverage measurement on the same codebase simultaneously
- Fetching and summarizing data from five independent sources at once
- Generating variations of the same artifact for A/B comparison
If the sequential execution time is acceptable and the subtasks have dependencies on each other, parallelism is not your limit. But when speed matters and the subtasks are genuinely independent, splitting into concurrent agents is structurally justified.
Test: Are there independent subtasks where waiting for each to finish before starting the next causes a real problem? If no, parallelism is not your limit.
An important caveat from the research: The DeepMind scaling study found that multi-agent coordination yields meaningful returns only when your single-agent baseline is performing below approximately 45% on the task. If your base model already handles the task well, adding parallel agents tends to introduce noise rather than improvement — the coordination overhead consumes the headroom that would have produced gains. Measure your single-agent baseline before splitting.
Limit 3: Specialization
Some tasks require fundamentally different capabilities, models, tool sets, or context that cannot coexist cleanly in a single agent.
Examples:
- A task that requires both a code-optimized model and a language-fluency-optimized model
- A task where one phase needs database read access and another needs deployment permissions — and you explicitly do not want both in the same execution context
- A task that requires domain-specific retrieval (e.g., legal documents) in one phase and general reasoning in another
Specialization is the rarest of the three limits in practice, because modern frontier models handle broad task ranges well. But when the capability gap between what one agent can do and what the task needs is real, specialization becomes the justification.
The permission context angle is the more important one in practice. In a bank, the employee who approves a payment and the employee who releases the funds are deliberately different people — not because one person could not do both jobs, but because having the same person do both removes the check. The separation is the safeguard. The same principle applies here: you split into separate agents not because a single agent lacks the capability, but because holding both privileges in one agent removes the audit layer between them.
Test: Does this task require capabilities, models, or permission contexts that are incompatible in a single agent? If no, specialization is not your limit.
The Architecture Ladder
If your task does not hit any of the three limits, stay at the bottom rung. Climb only when you have a specific, named limit that a higher rung solves.
┌─────────────────────────────────────────────────────────────┐
│ │
│ RUNG 4: Hierarchical / Mesh │
│ Multiple orchestrators, dynamic routing, swarm patterns │
│ Limit: Extreme scale, multi-domain specialization │
│ │
├─────────────────────────────────────────────────────────────┤
│ │
│ RUNG 3: Orchestrator-Worker │
│ One planner agent → N worker agents → result synthesis │
│ Limit: Context overflow + parallel independent subtasks │
│ │
├─────────────────────────────────────────────────────────────┤
│ │
│ RUNG 2: Pipeline │
│ Agent A → output → Agent B → output → Agent C │
│ Limit: Specialization across sequential stages │
│ │
├─────────────────────────────────────────────────────────────┤
│ │
│ RUNG 1: Single Agent │
│ One model, one tool set, one work loop │
│ Default starting point for every task │
│ │
└─────────────────────────────────────────────────────────────┘
Rung 1 — Single Agent
Start here. Always. Build the simplest version of the agent that could possibly work for the task. Give it the tools it needs. Observe where it breaks. Most tasks that feel complex are solved at this rung.
A well-designed single agent is:
- Easy to observe (one execution log, one output)
- Easy to debug (one prompt, one tool set)
- Easy to review (one artifact to inspect)
- Cheap to retry (one context to rebuild)
Do not graduate to a higher rung because the task is hard. Graduate only because you hit a named limit.
Rung 2 — Pipeline
A pipeline is a chain of specialized agents where each agent's output is the next agent's input.
┌──────────┐ ┌──────────┐ ┌──────────┐
│ Agent A │────▶│ Agent B │────▶│ Agent C │
│ (fetch) │ │ (analyze)│ │ (report) │
└──────────┘ └──────────┘ └──────────┘
Use a pipeline when the task has clearly sequential stages that benefit from different models, tool access, or permission contexts. A security scanning pipeline, for example, might separate evidence collection (broad read access) from patch generation (write access to a staging branch) to keep permissions scoped.
The key constraint: stages must be sequential. If they are not, you probably want Rung 3 instead.
Overhead added: Output handoff format between stages; failure propagation across the chain; harder end-to-end debugging.
Rung 3 — Orchestrator-Worker
An orchestrator decomposes a task into independent subtasks, dispatches them to worker agents, and synthesizes the results.
┌─────────────────┐
│ Orchestrator │
│ (plan + merge) │
└────────┬────────┘
│ decomposes
┌────────────┼────────────┐
▼ ▼ ▼
┌──────────┐ ┌──────────┐ ┌──────────┐
│ Worker 1 │ │ Worker 2 │ │ Worker 3 │
│ (subtask)│ │ (subtask)│ │ (subtask)│
└──────────┘ └──────────┘ └──────────┘
│ │ │
└────────────┴────────────┘
│ results
┌────────▼────────┐
│ Orchestrator │
│ (synthesize) │
└─────────────────┘
Use this pattern when the task genuinely exceeds context (each worker handles a scoped chunk) or when independent subtasks need to run in parallel for speed.
The orchestrator needs to be stable and well-specified. If the decomposition logic is fragile, every worker run is wasted. The synthesis step is also where results can silently contradict each other — build explicit reconciliation logic, not just concatenation.
Overhead added: Orchestrator prompt complexity; decomposition correctness requirement; synthesis logic; duplicated context setup across workers; higher total token cost.
Rung 4 — Hierarchical / Mesh
Multiple orchestrators, dynamic agent routing, and swarm-style coordination. This is genuine systems architecture territory — not a starting point. Most production engineering teams never need it.
Reach for Rung 4 only when you have multiple distinct domains, each of which already justified its own Rung 3 system, and those systems need to coordinate.
The Coordination Tax
Every rung above Rung 1 adds coordination overhead. This is not a flaw in the design — it is the structural cost of splitting work. The question is whether the benefit of splitting justifies the tax.
Research from Google DeepMind (Towards a Science of Scaling Agent Systems, Kim et al., 2025) puts concrete numbers on this cost:
- Unstructured multi-agent networks amplify errors by up to 17.2×. When agents operate without a formal coordination topology — a "bag of agents" where every agent communicates with every other — errors do not cancel out; they echo and compound. A centralised orchestrator architecture suppresses this to approximately 4.4×.
- Coordination gains plateau beyond ~4 agents. Adding more agents beyond this threshold produces diminishing returns on most tasks, and the coordination overhead begins to exceed the benefit.
- Sequential tasks degrade with every added agent. The study tested multi-agent systems on long-horizon planning tasks (Minecraft PlanCraft). Every multi-agent variant — independent, decentralised, centralised, and hybrid — degraded performance versus a single agent, with losses ranging from −39% to −70%. When the task is tightly sequential, agent coordination is pure overhead with no parallelism to offset it.
- Task type determines whether splitting helps at all. On highly decomposable tasks (financial analysis with independent subtasks), centralised MAS delivered +80.8% over single-agent. On dynamic web navigation tasks, centralised was flat (+0.2%) and independent MAS collapsed (−35%).
The coordination tax has five structural components that drive these numbers:
1. Error amplification without topology. In an unstructured system, each agent's errors become another agent's inputs. There is no circuit breaker. The 17.2× figure comes precisely from this: mistakes do not surface cleanly; they get passed forward, reinterpreted, and amplified at each step.
2. Handoff format brittleness. The output of one agent becomes the input of another. If the output format shifts — even slightly — the downstream agent breaks or silently misinterprets. Single-agent workflows do not have this problem.
3. Context duplication. Each worker agent needs enough context to do its job. Background knowledge must be re-injected into every worker context. The total token cost of a multi-agent run compounds with agent count — coordination rounds and peer messages inflate both message volume and context length.
4. Failure propagation. A single failing worker can corrupt the final result without surfacing a visible error. The orchestrator synthesizes what it receives. If a worker produced garbage quietly, the synthesis output is garbage with a confident tone.
5. Privilege creep and reviewability loss. Each agent in the system needs permissions. An orchestrator that spawns workers often needs to delegate permissions downward, expanding the attack surface. And where a single-agent run produces one artifact and one execution trace, an orchestrator-worker run produces N worker artifacts plus synthesized output — review time scales with graph size.
The Decision Checklist
Before adding a second agent to any system, answer all five questions:
□ 1. Does the task provably exceed context window limits?
If no → single agent handles it.
□ 2. Are there independent subtasks where parallel execution
materially reduces latency, and sequential is unacceptable?
If no → single agent handles it sequentially.
□ 3. Does this task involve sequential, tightly coupled reasoning?
If yes → do not split. Research shows every multi-agent
variant degrades performance on sequential tasks.
□ 4. Does the task require capabilities, models, or permission
contexts that are genuinely incompatible in one agent?
If no → single agent handles it.
□ 5. Is the single-agent baseline below ~45% on this task?
If no (base model already performs well) → more agents
will likely add noise, not improvement.
□ 6. Am I prepared to add handoff format contracts, failure
handling for each agent, and a synthesis review step?
If no → the system is not ready for the split.
□ 7. Can I explain in one sentence which specific limit
this split resolves?
If no → do not split yet.
If you cannot pass this checklist, Rung 1 is the right answer. Not because multi-agent is wrong, but because the cost is not yet justified.
Practical Starting Point
For any new task:
- Start with one agent. Give it the tools it needs. Run it. See where it breaks.
- Name the failure mode. Is it context overflow? Unacceptable serial latency? A capability gap? Or is it just slow or imperfect?
- Match the failure to a limit. Only context overflow, parallelism, and specialization justify climbing the ladder.
- Climb one rung at a time. Do not jump from Rung 1 to Rung 4. Each rung adds overhead. Verify the benefit before adding the next.
- Keep the review unit small. Regardless of which rung you are on, design the system so a human can inspect the work. If the output is not reviewable, the architecture is wrong for the risk level.
Summary
Multi-agent architecture is not an upgrade. It is a response to specific constraints. The three constraints that justify it are context overflow, parallelism, and specialization. Everything else — task difficulty, system complexity, the desire to feel more sophisticated — is not a justification.
Start at Rung 1. Stay there until one of the three limits forces you to climb. When you do climb, climb one rung, and pay the coordination tax consciously.
The teams that build durable agent systems are not the ones that added the most agents. They are the ones that knew when one was enough.
Primary research: Kim, Y. et al. (2025). Towards a Science of Scaling Agent Systems . Google DeepMind, arXiv:2512.08296. Additional sources: System Design One multi-agent architecture analysis; Cursor SDK documentation; Warp open-source release; The Neuron / Pi minimal-core agent design; Sean Moran, Why Your Multi-Agent System is Failing , Towards Data Science, January 2026.