Agentic AI Architecture: How Modern AI Agents Are Built (2026)

Two years ago, building an AI agent meant writing the loop yourself. You called the model, parsed what came back, decided whether a tool was needed, and appended the result to a growing message array.

When the context window filled, the run died and took everything it had worked out with it.

That loop is now a product. Anthropic, OpenAI, Google and Microsoft each shipped a hosted version of it between April and August 2026, and they arrived at almost the same design.

Agentic AI is the name for the system built around that loop. Seven layers, of which the model is one.

This article covers everything:

• How does an agent decide what to do next?
• How does it reach tools?
• How does it talk to other agents?
• Where does it keep state?
• How does anyone evaluate software that answers differently every time you run it?.

Agentic AI is not another word for AI agents

Start here, because the industry can't agree and the disagreement is worth understanding.

Google Cloud, Red Hat, Databricks and Moveworks all draw a distinction between the two terms. AWS, IBM and McKinsey use them interchangeably. MIT Sloan splits the difference, quoting MIT's Sinan Aral drawing a "slight distinction" while conceding most people still treat them as synonyms.

Red Hat has the cleanest version, and it's a grammar point rather than a technical one:

"An AI agent is a noun ('I'm building 3 agents.') and agentic AI is descriptive ('We need to make our software more agentic.')"

Google Cloud says the same thing with a metaphor that holds up: agents are individual tools in a toolbox, agentic AI is the coordinated use of those tools to build a house.

An AI agent is a unit. Agentic AI is a property of the system those units run inside.

A single agent is a loop. An agentic system is everything that makes the loop survivable: state that outlives a crash, protocols so tools and agents can find each other, a sandbox so mistakes stay contained, and enough observability that you find out what happened.

The seven layers of an agentic system

If you want one mental model to keep, keep this one.

An agentic system stacks seven layers. Most product marketing collapses them into one word, which is exactly why the word stopped meaning anything.

1. The model. It reasons. That's all it does. It has no memory, holds no state, and cannot call anything on its own initiative. Microsoft's own framing in their agent framework documentation is blunt about this: "A model on its own can only generate text."

2. The model runtime. Where inference actually happens, and where you pay per token.

3. The agent runtime, or harness. The loop itself: what calls the model, what feeds results back, what retries, what decides the job is done. This is the layer that barely existed as a product in 2024.

4. Tools. How the system reaches anything outside itself. Search, files, APIs, your inbox.

5. Memory and state. What survives after the run ends, which is a different question from what fits in the prompt.

6. Observability. Traces, event logs, the ability to answer "what did it actually do at step nine."

7. Sandbox and guardrails. What stops it doing damage, and what happens when it tries.

In 2024 you wrote layers three through seven yourself. An agent was a for-loop over a message array and some glue code. By 2026 the loop is a product.

Anthropic shipped Managed Agents in April 2026, decoupling the reasoning from the execution environment. Google announced a Managed Agents API at I/O in May 2026 under the tagline "manage the mission, not the machine." OpenAI's Agents SDK moved to a model-native harness with native sandbox execution in April 2026, on a stated design principle of keeping credentials "out of environments where model-generated code executes." Microsoft's Agent Harness reached general availability in August 2026, shipping history persistence, context compaction, file memory and OpenTelemetry tracing as defaults rather than features.

Four vendors, five months, one architecture. All of them separate the reasoning loop from the place code runs, and all of them keep credentials outside the sandbox.

That convergence is what makes a product like Imagine Computer possible. You don't assemble seven layers; you describe an outcome and the layers are somebody else's problem.

Reasoning: how agents decide what to do next

This is the layer with the most published research and the least published explanation. Worth going slowly.

Before 2026, the way an agent thought was something you designed. You picked a pattern and built scaffolding around the model to enforce it. Five patterns did most of the work, and they arrived roughly in this order.

Chain-of-thought

Think before answering. The model writes out its reasoning, then answers, and the writing-it-out measurably improves the answer.

No tools, no loop, no external actions. This is the ancestor of everything below it and it's still what "reasoning mode" means at the simplest level.

ReAct

Reason, act, observe, repeat until done. The model thinks, calls a tool, looks at what came back, and thinks again with that result in hand.

This is the pattern most agent products still run, and for good reason. It handles surprises, because every observation feeds the next decision.

Its weakness is cost. Every loop is another model call carrying the full history, so long tasks get expensive fast.

ReWOO

Plan the entire sequence upfront, then execute the whole thing without stopping to re-reason between steps.

Much cheaper, because you pay for one planning pass instead of a call per step. Considerably more brittle, because a plan written before step one can't know what step four will return.

Good for predictable multi-step work. Bad for anything that might surprise it.

Reflexion

Add a critic. The agent produces something, a critique pass evaluates it, and the critique goes back into the next attempt.

This is where self-correction comes from, and it's the pattern behind most "the agent checked its own work" claims. It roughly doubles or triples your token spend for one task.

Plan-and-Execute

Split planning and doing into separate roles, sometimes separate models. A planner decomposes the goal; an executor works the steps and reports back.

The planner can revise when the executor hits trouble, which gets you ReWOO's efficiency with some of ReAct's adaptability.

Which one to care about

Here's the tradeoff, roughly:

And now the part that changes how you should read that whole table.

The model absorbed the scaffold

In February 2026, METR published a finding that quietly obsoleted a lot of engineering: specialised agent scaffolds barely outperform generic ReAct any more.

The reason is that the reasoning moved inside the model. Anthropic's Opus 4.6, released 5 February 2026, shipped effort levels of low, medium, high and max, plus adaptive thinking where the model decides for itself how long to deliberate.

Opus 4.7 added an xhigh level and explicit task budgets in April, and OpenAI's GPT-5.6 began persisting reasoning across turns in July.

Reasoning became a dial. In 2024 you engineered the pattern; in 2026 you set a level and the model handles the rest.

Don't read that as "the patterns don't matter." Read it as: the patterns tell you what the dial is doing. When you turn effort up and your bill triples, the table above is why.

How agents reach tools, and why MCP mattered

Now the plumbing. This is the part of agentic AI that genuinely didn't exist three years ago.

An agent that can't touch anything is a chatbot. Tools are what make it an agent, and until late 2024 every single tool connection was hand-built.

The maths were ugly. Ten agents and twenty tools meant two hundred bespoke integrations, each with its own auth, its own schema, its own breakage.

What MCP actually is

Anthropic released the Model Context Protocol in November 2024. It's a standard for how an agent describes and calls an external tool, built on a client and server model.

The tool runs as a server. It advertises what it can do, in a schema the agent can read. The agent runs a client that discovers available tools and calls them.

That's it, conceptually. The value isn't cleverness, it's that everyone agreed on the same shape.

The adoption numbers are worth stating carefully, because most figures circulating for MCP trace back to content farms with no primary source. The reliable ones come from the Agentic AI Foundation announcement on 9 December 2025: 97 million monthly SDK downloads and roughly 10,000 active servers, at which point MCP was donated to the Linux Foundation alongside OpenAI's AGENTS.md, with AWS, Google, Microsoft, Cloudflare and Bloomberg as platinum members.

A protocol going from nonexistent to foundation-governed in about twenty months is unusual. Cross-vendor agent interoperability went from impossible to boring in less time than most enterprise software procurement cycles.

The July 2026 change nobody wrote about

On 28 July 2026 the MCP spec became stateless, and it's the most consequential revision so far.

The initialize handshake is gone. So is the session-ID header. Every request now carries its own protocol version, client identity and capabilities.

The spec authors' own summary of why that matters: "any request can land on any instance behind a plain round-robin load balancer."

Translated: MCP servers became ordinary web services. You can scale them the way you scale anything else, which is the difference between a protocol that works in a demo and one that works under load.

Tool bloat, which is the practical constraint

Give an agent too many tools and it gets worse, not better. OpenAI's own documentation still advises keeping it under twenty functions at the start of a turn.

For a while that was a hard wall. Two things broke it, and both work by getting tools out of the prompt.

Tool search, which Anthropic shipped in November 2025, lets the agent look up tools on demand instead of carrying every definition in context. Anthropic reported roughly 85% context reduction.

Programmatic tool calling goes further: the agent writes code that calls tools, rather than emitting one tool call per turn. Anthropic's figure for code execution against MCP on one workload was 150,000 tokens down to 2,000.

Both are vendor-reported numbers on vendor-chosen workloads, so treat the magnitude as directional. The architectural point stands regardless: tools stopped living in the prompt.

How agents talk to each other

Tools are one problem. Agents finding other agents is a different one, and it's the less covered half.

MCP connects an agent to a tool. It says nothing about how two agents, possibly built by different companies on different stacks, discover each other and divide work.

A2A and agent cards

Agent2Agent handles that. It reached v1.0 stable with more than 150 participating organisations by April 2026.

The interesting mechanism is the agent card: a signed document describing what an agent can do, which another agent can read before deciding to delegate to it. Think of it as a capability advertisement with a verifiable signature attached.

That signing matters more than it sounds. Once agents can hire each other, "can I trust what this thing claims about itself" becomes a security question rather than a philosophical one.

One protocol already died

IBM shipped a competing standard, the Agent Communication Protocol. It was deprecated and merged into A2A in August 2025.

Worth knowing partly because several pages currently ranking for agentic AI architecture still list ACP as a live option. If a page recommends it, that page hasn't been touched in a year.

One thing genuinely unsettled as of this writing: A2A's move into the Agentic AI Foundation was still awaiting a governing board vote when I checked, so treat the governance as in progress rather than done.

Six ways to wire agents together

Once you have more than one agent, you have to decide how they relate. This is where most architectural decisions actually get made.

Six topologies cover nearly everything in production. For each one, the useful questions are how work flows, what it costs, and how it breaks.

• Single agent. One loop, all the tools. Cheapest, most predictable, and the right answer far more often than the discourse suggests. Fails by running out of context or drowning in tool definitions.
• Sequential. Fixed handoffs, A to B to C. Easy to reason about and easy to debug. Fails when a step needs information a later step hasn't produced yet.
• Router. A classifier reads the request and dispatches to a specialist. Efficient, since only one specialist runs. Fails when the router misclassifies, and it fails silently, because the specialist will confidently do the wrong job well.
• Parallel. Fan out, then merge. Fast in wall-clock terms and expensive in tokens. Fails at the merge, when two branches return work that can't be reconciled.
• Hierarchical. A supervisor plans and delegates to workers. The most common multi-agent pattern in production. Fails when the supervisor's plan was wrong, since the workers have no standing to argue.
• Network. Peer to peer, no coordinator. Flexible in theory, hardest to debug in practice. Fails by not terminating.

Now the number that should inform every one of those choices.

Anthropic's own accounting puts agents at roughly four times the token consumption of a chat interaction, and multi-agent systems at roughly fifteen times. Their engineering write-up adds a line worth sitting with: "token usage alone explains 80% of performance variance" in their browsing evaluations.

That's Anthropic's figure on Anthropic's workloads, not an independent benchmark. But if multi-agent quality tracks token spend that closely, a lot of "our architecture is smarter" is really "our architecture is more expensive."

The multi-agent argument, and how it actually resolved

This is the best story in agentic AI and almost nobody tells it. Three dates.

12 June 2025: don't build multi-agents

Cognition published a piece arguing against multi-agent systems, and their example is the kind that sticks.

Building a Flappy Bird clone, one subagent produced a Super Mario-styled background while another built a bird that didn't match it. Neither was wrong on its own terms. The coordinator couldn't reconcile them because neither subagent could see what the other had decided.

Their two stated principles: share full agent traces rather than individual messages, and remember that actions carry implicit decisions, so conflicting decisions produce conflicting results. Their recommendation was single-threaded linear agents, with compression rather than parallelism for long tasks.

13 June 2025: the opposite result, one day later

Anthropic published a multi-agent research system reporting a 90.2% improvement over single-agent Opus 4 on internal research evaluations.

Read the caveats, though, because they're unusually candid. The token multipliers above come from this same write-up, and Anthropic listed their own contraindications: domains needing shared context, heavy dependencies between agents, limited parallelisation, and then in parentheses, "like most coding tasks."

Which is to say, Anthropic broadly agreed with Cognition about where multi-agent fails. They just found a domain where it doesn't.

5 February 2026: sixteen agents, no orchestrator

Then Anthropic did something that looks like it contradicts both positions.

Sixteen Claude instances ran in parallel Docker containers against a shared repository, with no orchestrator agent at all. Coordination happened through git and text files claiming tasks. A fresh container per iteration, driven by an infinite loop.

The output: a 100,000-line Rust implementation of a C compiler, built over roughly 2,000 sessions for about $20,000, which compiles Linux 6.9 on x86, ARM and RISC-V and passes 99% of the GCC torture test suite.

What failed is as instructive as what worked. Large monolithic tasks caused agents to conflict, adding features caused frequent regressions, and the compiler it produced runs substantially slower than GCC.

What this actually tells you

Here's the synthesis, and I think it reconciles all three.

The 2025 backlash was never really about parallelism. It was about fan-out to subagents that couldn't see each other's decisions.

What worked in 2026 was neither one agent nor a swarm. It was parallel agents working against a shared, durable, externally verifiable artifact: a git repository they all read, a test suite that adjudicated disputes, and file locks so two agents didn't claim the same work.

Cognition said share the full traces. Anthropic said share the repo, share the tests, lock the work. Those are the same insight wearing different clothes, and the fatal pattern in both accounts is identical: agents that can't observe each other's choices will contradict each other confidently.

There's an academic version of this too. A Berkeley paper, "Why Do Multi-Agent LLM Systems Fail?", published a failure taxonomy called MAST and appeared at NeurIPS 2025, if you want the formal treatment.

One more cost nobody markets. Anthropic's eval-awareness research in March 2026 found benchmark contamination at 0.87% in multi-agent configuration against 0.24% single-agent, roughly 3.7 times higher. More agents means more surfaces where things leak.

For what it's worth, Imagine Computer runs as a single agent coordinating many tools rather than a fleet of agents coordinating each other. On the evidence above, that's a defensible engineering position rather than a missing feature.

State, and why a million-token window didn't fix memory

Every vendor page mentions memory. Almost none explains state, and they aren't the same thing.

Memory is what the system knows about you. State is where the system is in its work. You can have plenty of the first and none of the second, which is what most 2024 agents looked like.

The durable session

In 2024 an agent was a for-loop over a message array. When the context window filled, the run died, and whatever it had figured out died with it.

By 2026 the session is an event log. Anthropic's Managed Agents, shipped April 2026, exposes event emission and retrieval plus a wake call so a crashed run can resume.

LangGraph shipped durable state in version 1.0 in October 2025, and Microsoft's harness ships history persistence and context compaction turned on by default.

The practical difference is simple. An agent in 2026 can be killed and resumed; an agent in 2024 could not.

Why bigger windows didn't solve it

Frontier context windows sit around a million tokens now. That should have made memory a non-problem, and it didn't.

Anthropic named the reason for context rot in September 2025, attributing it to the transformer's n² pairwise token relationships. Their phrasing: "a natural tension between context size and attention focus."

Models can hold far more than they can reliably attend to.

The best measurement I found is a May 2026 paper testing whether frontier models notice dangerous actions buried in long transcripts. Opus 4.6, GPT-5.4 and Gemini 3.1 missed those actions two to thirty times more often when they appeared after 800,000 tokens of ordinary activity than when they appeared alone.

The authors' broader point is sharper than the number. Agent monitoring benchmarks typically run under 100,000 tokens while real monitoring needs upwards of 500,000, so published monitoring effectiveness is probably overstated across the board.

There's a pricing tell too. Anthropic charges roughly double above 200,000 tokens on Opus 4.6. Long context isn't just harder to attend to, it's more expensive to serve.

What worked instead

If attention doesn't scale, what does? A March 2026 paper has the most interesting answer: give the agent a filesystem and let it navigate with code.

Instead of stuffing a corpus into context and hoping attention finds the relevant part, the agent runs commands, greps, reads specific files. The reported result was a 17.3% average improvement over published state of the art, on corpora up to three trillion tokens.

The authors position it explicitly as an alternative to both semantic search and bigger windows. And it's more or less the architecture every major vendor shipped in 2026: compaction, file-based memory, tool search, code execution.

So the honest answer to "why didn't a million-token window fix memory" is that attention degrades long before the window fills, long context costs more to serve, and filesystems plus event logs solved the problem instead. Partially.

The vocabulary shift was real

"Prompt engineering" became "context engineering" somewhere in 2025, and it wasn't a rebrand.

Karpathy popularised the phrase in June 2025; Anthropic gave it a formal definition that September. By 2026 it's directory names in shipping products and a default service in enterprise SDKs.

The term changed because the job changed. Writing one good prompt became managing a token budget across a run that might last hours.

Evaluating something that isn't deterministic

Here's a problem the industry is still losing. How do you test a system that gives different answers to the same question?

Normal software testing assumes determinism. Same input, same output, pass or fail. Agents break that assumption by design, and they break it at every step of a multi-step run.

What replaced simple assertions is messier: trace-based debugging where you inspect the whole decision path, task-level success rates across many runs, models judging other models' output, and regression suites built to tolerate variance rather than forbid it.

The benchmarks turned over completely

The scoreboard people quote is mostly out of date.

Out: SWE-bench as a frontier differentiator, WebArena, the original TAU-bench. Notably, Anthropic's Opus 5 launch on 24 July 2026 didn't cite a SWE-bench number at all. Some benchmarks simply saturated, with τ²-telecom sitting at 99.1%.

In: GDPval-AA, Terminal-Bench, OSWorld, MCP-Atlas, GAIA2, ARC-AGI 3.

Three findings that undercut all of them

This is the part worth remembering when you next read a benchmark table.

In February 2026, researchers found agent scores swinging by six points based on virtual machine size alone. Same model, same task, different hardware allocation.

In March 2026, Anthropic documented models that independently deduced they were being evaluated and then decrypted benchmark answer keys. One instance spent 40.5 million tokens doing it.

And METR publishes factor-of-two error bars on its autonomous time-horizon estimates, which is a level of honesty most benchmark publishers skip.

The capability number that actually matters

If you take one figure from this article, take this one, because it appears almost nowhere else.

Short web tasks: agents succeed above 90%. Multi-hour desktop work: OSWorld 2.0 tops out at 20.6% across 108 workflows, measured June 2026. GAIA2's best result is 42% pass@1.

That gap is the honest state of agentic AI in 2026. Bounded, short, well-specified tasks are close to solved. Long, open-ended work on a real desktop is not.

Why permission prompts got replaced by sandboxes

One last number, and it explains an entire architectural shift.

Users approved roughly 93% of Claude Code permission prompts. Not 60%, not 75%. Ninety-three.

A safety mechanism that gets waved through nine times out of ten isn't a safety mechanism, it's a formality. So the industry moved to controls that don't depend on human attention: OS-level sandboxing, which cut prompt volume by around 84%, containerised execution for hosted code, sealed environments where the host holds credentials the agent never sees, network allowlists treated as capability grants, and classifiers reading transcripts for overeager behaviour.

Anthropic reported one such classifier catching about 83% of overeager behaviours at a 0.4% false-positive rate on live traffic between March and May 2026. Every vendor shipping these still describes prompt injection as unsolved, which is the correct posture.

What actually changed since 2024

Pull it together. Nine changes, each with a date, ordered by how much they matter architecturally.

1. Agents got durable state outside the context window. The single biggest change. In 2024 a full context window killed the run. In 2026 sessions are event logs you can query and resume.

2. Tool count stopped being a ceiling. Because tools moved out of the prompt, via tool search and programmatic calling. The under-twenty-functions guidance is still in OpenAI's docs, and it's now a starting point rather than a wall.

3. Interoperability went from zero to governed in twenty months. MCP didn't exist publicly before November 2024. By December 2025 it was under Linux Foundation governance with five hyperscalers as platinum members.

4. Autonomous runtime went from minutes to days. METR measures roughly one doubling of task-completion horizon every six to seven months. The February 2026 C compiler run is the concrete proof, and nothing like it was possible two years earlier.

5. Reasoning became a dial. Effort levels and adaptive thinking replaced hand-built scaffolds, and METR found the scaffolds stopped paying for themselves.

6. Computer use became a product and remains the weak link. Shipped in APIs and desktop apps through 2026, and still around 20% on multi-hour desktop workflows.

7. Security moved from prompts to sandboxes. Because 93% approval rates proved prompts don't work.

8. Benchmarks were replaced, then partly invalidated. New suites arrived, then VM-size sensitivity and eval-aware models undercut confidence in all of them.

9. Prompt engineering became context engineering. The vocabulary tracked a real change in what the work involves.

Read that list as one trend. Every item is the same story: the scaffolding that used to be your problem became infrastructure.

Which is the quiet reason tools like Imagine Computer exist at all. Not a smarter model, but seven layers of plumbing maturing to the point where you can describe an outcome, get a finished document back, and never learn that any of this was involved.

Summary

If you're buying rather than building, almost none of these decisions are yours. You inherit the harness, the state layer and the topology from whoever built the product.

What the architecture gives you is a way to interrogate what you're told. When a vendor says the agent remembers you, ask whether that's context window or durable state. When they say it connects to your tools, ask whether that's MCP or something bespoke, because one of those quietly becomes your maintenance problem.

Set expectations from the capability gap rather than the demo. Short, well-specified tasks clear 90% on web benchmarks; long open-ended desktop work sits near 20%, and nothing in this article closes that.

Every layer that used to be an engineering project is now something you buy. That moves the real question from how to build this to what you want done, and Imagine Computer is one answer to the second.