Every vendor deck I have been sent this year has agents in it. Most of what they describe is a chatbot with better marketing, and a few are workflow tools that have discovered a new adjective. This matters more than it sounds, because the distinction is not academic — it decides how you budget, what you build, and whether the thing survives contact with your business.
So let me set out the line we use internally, and where the real work sits.
The distinction that actually matters
Forget architecture diagrams for a moment. The practical test is this:
Can it change something in one of your systems?
If the answer is no — if it can only tell you things — you have a chatbot, however sophisticated its reasoning. That is not an insult. Retrieval chatbots are excellent, cheap, and often exactly the right answer. But their ceiling is information.
If the answer is yes — it can book the meeting, update the record, raise the ticket, submit the claim — you have an agent. And the moment you cross that line, the engineering problem changes completely. A chatbot that is wrong wastes someone’s time. An agent that is wrong does something wrong, in a system of record, on your behalf.
Everything hard about agent development follows from that sentence.
Why “multi-step with judgement” is the real qualifier
The other half of the definition is about the shape of the work. Agents earn their cost on tasks that are:
- Multi-step — the work needs several actions in sequence, and the right next step depends on what the last one returned.
- Variable — the inputs are not identical every time. A form is not variable. A customer’s email describing a problem is.
- Judgement-bearing — somewhere in the middle, someone currently decides something.
Take any three of those away and you probably do not need an agent. I want to be blunt about this, because it is where money gets wasted. If your process is high-volume and identical every time, rule-based automation will be cheaper, faster, and more reliable than any agent, and it will not surprise you at 3am. Reach for the LLM when the variance is real.
The tasks that fit the profile tend to look mundane written down: triage the inbound queue, reconcile these two records, prepare the first draft of this report, qualify this lead against our criteria, check whether this claim needs a human. Mundane is good. Mundane is where the hours are.
What the work actually consists of
Here is the thing that surprises people who have only seen agent demos. Building the agent — the prompt, the loop, the tool definitions — is a small fraction of the effort. In our experience, the majority of the work sits in three places, and skipping any of them is how you get a project that demos beautifully in week three and is quietly abandoned by month five.
1. Grounding
Language models are excellent at producing plausible sentences and indifferent to whether those sentences are true of your business. An agent with no grounding will confidently describe a product feature you do not have, quote a policy you never wrote, and invent an integration that does not exist.
The fix is not a better prompt. It is retrieval: the agent’s answers come from your documents, databases, and APIs, fetched at the moment of the question. And crucially, a floor beneath it — when the retrieval finds nothing relevant, the agent must decline rather than fall back on the model’s general knowledge.
That last part is the bit most teams miss, and it is the single most valuable behaviour you can build in. An agent that says “I don’t have anything on that, here’s who to ask” earns more trust in one interaction than a hundred fluent guesses.
2. Evaluation
You cannot improve what you do not measure, and with agents you cannot even tell whether you have broken something. Change a prompt to fix one behaviour and you have silently changed a hundred others.
So before building the agent, build the test set. Twenty to fifty real cases from your actual process, with known-good outcomes. Then measure every change against them:
- Task success — did it complete the job?
- Groundedness — was every claim traceable to retrieved content?
- False refusals — did it decline things it should have handled?
- Latency and cost — per task, because this is what determines whether the economics work at volume.
This is unglamorous and it is the difference between engineering and guessing. Teams that skip it end up in an endless loop of prompt-tweaking based on whichever failure someone noticed most recently.
3. Scoped tools and permissions
An agent should be able to do exactly its job and nothing more. In practice that means: each tool takes validated inputs, the credentials behind it are scoped to the narrowest possible permission, destructive actions require confirmation, and there is a human approval step at whichever points a wrong action would actually cost you something.
Deciding where those points go is a business conversation, not a technical one. It is also the conversation that determines whether your team trusts the thing. Put approval everywhere and you have built an expensive draft generator. Put it nowhere and nobody will sign off on launch. The right answer is specific to your risk, and it is worth the argument.
The failure modes we see most
Pointing an agent at work that never needed one. The most expensive mistake, and the most common. Usually visible in the first workshop if anyone asks how often the inputs actually vary.
No relevance floor. The agent answers everything, because nothing tells it not to. Fluent, confident, occasionally wrong in ways that are hard to spot — which is worse than being obviously wrong.
Tool sprawl. Fifteen tools because each one seemed useful. The agent’s reliability drops with every option it has to choose between, and debugging becomes archaeology.
Demo-driven development. The build optimises for the scenario that gets shown to stakeholders. Reality has a much longer tail, and the tail is where adoption is won or lost.
No observability. Something goes wrong in production and there is no trace of what the agent retrieved, what it decided, or which tool it called. You cannot fix what you cannot see, and with a multi-step agent you need the whole chain, not just the final output.
How to tell whether you have a real candidate
If you are trying to work out whether agents are relevant to you, the useful questions are not about technology:
- Name a task that takes your team more than ten minutes and involves more than one system.
- How much do the inputs vary, case to case? If the honest answer is “not much”, automate it deterministically instead.
- Where in that task does someone currently exercise judgement? That is where your human approval step goes.
- What would you measure to know it was working? If you cannot answer, that is the first piece of work.
- What happens if it gets one wrong? The answer sets your guardrail budget.
If you can answer those five, you have a scope. If you cannot answer the fourth, start there — not with a model.
Where this leaves the hype
Agents are not overhyped in the sense of being useless. They are overhyped in the sense that the label has been stretched across everything, and the genuinely hard parts — grounding, evaluation, permissions, observability — are the least demo-able and therefore the least discussed.
Our own site assistant is built exactly this way, deliberately. It retrieves from our published content, declines anything outside its remit, and books meetings on a real calendar. It is narrow on purpose. Ask it something off-topic and it will politely refuse — that refusal is not a limitation we are apologising for, it is the feature that makes the rest of it trustworthy.
If you are weighing up agent work, the most useful thing you can bring to a first conversation is not a technology preference. It is a task. We can usually tell you within half an hour whether an agent is the right shape for it — and we will tell you when it isn’t.
If you want to go deeper on how we build these, our AI agent development page sets out the process, the stack, and what it costs. Or talk to us about a specific task.
info@xpergia.com 