Agentic architectures compared
An “agent,” in the sense people mean when they talk about agentic AI, is just a language model given a tool and allowed to decide for itself, turn by turn, what to do with it: search again, ask someone else, or answer. The interesting design question isn’t whether to build one. It’s how to structure the deciding. Do you let one model make every call itself? Do you write out a fixed plan ahead of time and just execute it? Do you let a coordinator revise the plan as new facts come in? Do you add a second role whose only job is to check the first role’s work? Every one of these shows up in real products right now, usually described in a blog post that builds exactly one of them and never puts it next to the alternatives. I wanted to build all five, on the same task, and see where each one actually earns its complexity and where it just adds cost.
Try the live dashboard or read the full write-up and code.
The setup
I used HotpotQA, a multi-hop question answering dataset where every question needs facts from two different Wikipedia paragraphs, not one, to answer. Each question ships with 10 candidate paragraphs, 2 that actually support the answer and 8 distractors that don’t. That fixed set of 10 becomes the corpus a single search() tool looks through for that question, and the tool always returns just its single best-matching paragraph. With 2 paragraphs needed and only 1 returned per search, no architecture can get away with a single lookup. Every one of them has to decide, somehow, whether and how to search again, which is exactly the design question this experiment is about.
Every architecture ran on the same local, free model (llama3.1:8b through Ollama), against the same 40 sampled questions, split evenly between two question types HotpotQA labels itself: bridge questions, where the second lookup needs a name or fact the first lookup turns up, and comparison questions, where the two lookups are independent of each other.
flowchart LR
A["HotpotQA, 40 sampled questions"] --> B{"5 architectures"}
B --> C["llama3.1:8b, same model for all"]
C --> D["search() over that question's 10 paragraphs"]
D --> E["An answer, plus how it got there"]
Five ways to structure a multi-step agent
All five architectures are built out of the same two ingredients: one language model, and one tool. What differs, architecture to architecture, is only who decides what to search for next, and who decides when enough has been found. That second question turned out to matter far more than I expected going in.
The simplest version is Single-Agent ReAct, named after the pattern Yao et al. described in 2022: one model, one running transcript, no other roles. Each turn it writes a short thought about what it’s doing and why, then either searches again or answers, and whatever the search returns gets folded back into the same transcript before the next turn. There’s no planner deciding the shape of the work in advance and no second opinion checking the result; the model that reasons about the question is the same model that decides when it’s reasoned enough. That’s also its whole appeal: it’s the cheapest thing to build, the easiest to debug, since there’s only one transcript to read top to bottom, and it doesn’t need you to know the shape of the task ahead of time. The catch is that “when it’s reasoned enough” is exactly the judgment call it turns out to be worst at. In my results it flat-out never answered on over a third of its runs, not because it ran out of useful things to try, but because it kept repeating a search that had already failed rather than committing to whatever it already had.
Sequential Pipeline takes the opposite bet: decide the whole plan before doing any of the work, then execute that plan in a fixed order every time. A decomposer writes out both sub-questions the moment it sees the original question, then each one gets searched and answered in turn, then a final step combines both answers. Nothing ever revisits an earlier stage once it’s run. This is the right shape when you already know your subtasks don’t depend on each other, comparison questions being the clean example, because then a fixed plan costs the least and is the easiest to test one stage at a time. The pitfall is baked into the design on purpose: the plan is written blind, before either lookup has happened, so if the second sub-question actually needs a name only the first lookup would surface, the decomposer has to guess that name before it exists anywhere in the transcript. I watched this fail on exactly that kind of question: asked what 2001 film a director’s debut was a sequel to, the fixed plan guessed “Top Gun,” a real movie, just not the right one, because it had no way to know the right one until the first lookup had already run, and by then it was too late to rewrite the question that needed it.
Orchestrator, adaptive dispatch is the fix that idea is begging for: instead of writing the whole plan upfront, a coordinator looks at one sub-question at a time, sees what came back, and only then decides the next one, so it can fold in a name the first lookup just found. This is supposed to be strictly better on exactly the kind of question that breaks the fixed pipeline, and on paper it is: it can use information the fixed pipeline structurally can’t. What I found instead is a different, quieter failure. Across all 40 runs, the coordinator never once decided on its own that it had gathered enough. Every single run only stopped because I’d capped it at 3 rounds and it hit that cap; left to its own judgment, it kept generating another sub-question indefinitely, sometimes reasonable, sometimes redundant with something it had already answered two turns earlier. That’s a genuinely different problem from getting the wrong answer, and a more dangerous one to miss, because a pure accuracy score would never surface it. An architecture that’s rescued by a hardcoded limit every single time isn’t adaptive in the way its design promises; it’s just an unbounded loop with a leash on it.
Orchestrator, parallel dispatch keeps the fixed pipeline’s upfront plan but runs both lookups at the same time instead of one after another, on the reasoning that if two sub-questions are genuinely independent, there’s no reason to make one wait for the other. When that reasoning holds, this is close to a free win: same answer, less wall-clock time. It has two ways to fail, though, and they’re worth telling apart. The structural one is the more interesting: run two things in parallel and neither one can ever see the other’s result, so if a task actually has a dependency between its steps, parallel dispatch doesn’t handle that case poorly, it makes handling it impossible by construction, a stronger and more absolute failure than the fixed pipeline’s blind guess. The second is more mundane and easy to miss when you’re only reading the architecture diagram: the speed benefit is a promise about your serving infrastructure, not about your code. Running everything through one local model server that only processes one request at a time, I measured almost none of the expected speedup, because two “concurrent” requests from two threads were still just queued up and answered one at a time underneath. The architecture was doing its job; the infrastructure underneath it wasn’t built to reward that job.
Supervisor with a verification loop is the most structurally different of the five: instead of one coordinator dispatching interchangeable workers, it splits the work across three specialists with genuinely different jobs, a retriever that fetches evidence, a reasoner that drafts an answer from it, and a verifier whose only role is to check whether that draft is actually backed by what was fetched, with no stake in being right. A rejected draft sends the supervisor back to try a different search, informed by exactly what the verifier said was missing, rather than accepting whatever the first pass produced. This is the reflection idea behind papers like Reflexion and Self-Refine, given a dedicated role for the skepticism instead of asking one model to grade its own homework in the same breath it wrote it. It’s also, in my results, the one architecture that pulled meaningfully ahead on accuracy, and specifically by catching answers that sounded fine but weren’t actually supported by anything retrieved, not by reasoning its way through the hardest multi-hop chains. The catch is that a verifier can only reject a bad draft; it can’t manufacture better evidence than what’s actually out there. When the underlying search turns up nothing better no matter how the query gets reworded, “try a different phrasing” just produces the same evidence again with different words, and the loop burns its whole round budget rejecting drafts without ever finding one to accept. In my data, roughly half of its runs ended exactly that way.
How I measured this
Three different questions need three different kinds of measurement, and I found out the hard way that mixing them up hides most of what’s actually interesting. Quality metrics ask whether the final answer was right, exact match and a partial-credit F1 score against the correct answer. Agentic metrics, the category built specifically for this experiment, ask how the architecture got there: how many steps it took, how many times control passed between roles, and critically, whether it stopped because it decided to or because a hard limit forced it to. Operational metrics ask what all of that cost: tokens, latency, and how often a search came back with a paragraph that wasn’t actually one of the two that mattered.
That middle category turned out to be the one doing almost all the real work. Four of the five architectures tied at 47.5% exact match. Read only that number and the honest conclusion is that architecture barely matters here. It’s the metric measuring how each one stopped, not whether it got the answer right, that shows they were behaving nothing alike underneath an identical score.
What actually happened
| Architecture | Exact match | Never stopped on its own | Mean steps |
|---|---|---|---|
| Single-Agent ReAct | 47.5% | Never finished 35% of runs | 6.7 |
| Sequential Pipeline (fixed) | 47.5% | 0% | 6.0 |
| Orchestrator (adaptive) | 47.5% | 100% | 10.0 |
| Orchestrator (parallel) | 47.5% | 0% | 6.0 |
| Supervisor + Verification Loop | 60.0% | 47.5% | 7.0 |
A few things were worth digging into beyond that headline number.
The verification loop won by catching bad answers, not by out-reasoning the hardest questions. Its accuracy lead is almost entirely on the independent, comparison-style questions (a full 20-point jump there), and it basically disappears on the genuinely chained, multi-hop questions, where it’s statistically indistinguishable from everything else. The mechanism was visible in how its rounds played out: most runs either got accepted on the very first draft, or exhausted all 3 rounds rejecting every draft, hardly any landed in between. Refinement rescues a mediocre first search. It doesn’t invent evidence that was never there to begin with.
More coordination between roles didn’t mean a better result, and spending more tokens didn’t either. The adaptive orchestrator and the verification loop both average the same number of handoffs between roles, the most of any architecture here, for the worst accuracy and the best accuracy respectively. The adaptive orchestrator also spends the second-most tokens per question for tied-worst accuracy. Both numbers only mean something once you read them against the outcome; neither one predicts it alone.
Parallel and fixed-sequential dispatch gave byte-for-byte identical answers on every single question. That’s expected once you think about it: with the model always picking the single most likely next word, running the same two lookups in a different order changes nothing about what either one finds. The only real difference between them was speed, and as described above, my local server didn’t let that difference show up the way a properly parallel backend would have.
Where this leaves me
Of the five, HotpotQA gave three of them a genuinely fair, even generous, shot at showing their strength: the fixed pipeline on questions whose two parts really are independent, the adaptive orchestrator on questions that are the textbook case for re-planning, and the verification loop on a task with exactly the kind of clean evidence a skeptic role needs to check against. Two of them didn’t get that same fair shot, for reasons that have nothing to do with which questions I picked. Parallel dispatch’s entire value is wall-clock savings from real concurrency, and that depends on the server underneath it, not on the dataset. A single-agent’s real advantage is adapting its own step count to a task whose length you can’t know in advance, and HotpotQA’s hop count sits at almost exactly 2 throughout, so that flexibility never actually gets asked for.
That points fairly directly at what I’d want to test next: a task whose step count genuinely varies, to find out whether an agent’s stopping problem is something about this specific 8-billion-parameter model or something more fundamental to the pattern itself, and at least one run against real hosted, batching-capable infrastructure, to see what parallel dispatch’s latency case actually looks like when it’s given the chance to be true.
The full write-up has every architecture’s exact prompts, a working demo where you can inspect any single run’s trace turn by turn, and the honest caveats (40 questions is enough to see a clear directional difference, not enough to trust an exact percentage-point gap; a local 8-billion-parameter model run at temperature 0 is a noisier narrator than a larger hosted one would be, and some of what looks architectural here is really about how reliably this specific model follows instructions).
Read the details, or explore the dashboard yourself.