Building Effective AI Agents: Start Smaller Than You Think
The first decision when building an AI feature is not which agent framework to choose. It is whether the feature needs an agent at all.
Anthropic’s guide, Building Effective AI Agents, makes a useful distinction between workflows and agents:
- A workflow follows a defined path. The application controls the steps.
- An agent decides how to complete a task. The model controls parts of the process and chooses tools as it goes.
The practical advice is to begin with the simplest system that can meet the requirement. Simple systems cost less, are easier to debug and give clearer product metrics.
The useful patterns
Prompt chaining
Break a task into a sequence of model calls. The output from one step becomes the input to the next.
This works when the stages are known: extract data, classify it, then produce a response. Add a validation gate between steps when a bad intermediate result would make the final answer unreliable.
Routing
Classify the request, then send it to a specialist prompt, model or workflow. Routing keeps a simple path cheap while allowing difficult requests to reach a more capable path.
It is a good fit for support systems where account questions, technical issues and complaints need different instructions or permissions.
Parallelisation
Run independent checks at the same time and combine their results. This can improve latency or give several perspectives on one problem.
It is useful for guardrails, review and risk analysis. It is a poor fit when each step depends on the previous one or when agents would update shared state at the same time without conflict handling.
Orchestrator and workers
An orchestrator breaks a broad task into pieces, delegates them to specialists and combines the results. The workers can stay focused on their domain while the orchestrator owns the overall answer.
This pattern is powerful, but the coordinator can become a context and reliability bottleneck. Keep worker outputs structured and bounded; do not make the orchestrator carry every transcript forever.
Evaluator and optimiser
One model produces an answer and another evaluates it against explicit criteria. The generator then revises the answer and the cycle repeats.
This is useful when quality can be judged clearly, such as code generation, documentation or regulated content. It adds latency and cost, so I would not use it for a task that already meets its quality bar in one pass.
Autonomous agents
An autonomous agent chooses its own sequence of actions and keeps going until it reaches a stopping condition. This is the most flexible pattern and the one that needs the strongest guardrails.
Give it narrow tools, bounded budgets, clear success criteria and a way to hand work back to a person. “Keep trying” is not a stopping condition.
How architectures evolve
I would not start with a multi-agent system because the diagram looks impressive. Start with a focused agent or workflow, measure the result and add complexity only when the evidence points to a real limitation.
single call
-> chained workflow
-> routed workflow
-> parallel or evaluator loop
-> delegated or multi-agent system
That progression is not a maturity ladder. A simple route may be the right final architecture. A multi-agent system is justified when the problem needs independent expertise, dynamic delegation or work that cannot fit comfortably in one context.
The production checklist
Before calling a system effective, I want to know:
- What outcome does it own?
- Which steps are deterministic and which are model-controlled?
- What tools can it call, and what can each tool change?
- How do we measure quality, cost and latency?
- What happens when retrieval is empty, a tool fails or the model loops?
- Where does a person approve, correct or take over?
An agent is effective when it solves a valuable problem reliably enough to operate. The architecture should make that easier, not hide the uncertainty behind more boxes.