An impressive demo does not establish that an agent is ready for real users. Production introduces incomplete data, unexpected requests, and operational failures no script anticipated. The model was never the hard part — everything around it is.
This checklist covers the engineering disciplines that separate a demo from a system you can trust in front of customers. None of it is exotic; all of it is skipped more often than it should be.
Ground the agent in real inputs before choosing architecture
Build your evaluation set before you build the agent. Collect actual questions, conversations, or tasks — fifty honest examples beat five hundred hypothetical ones. This set becomes the thing you test every change against, and it will immediately show whether you need retrieval, tools, fine-tuning, or just a well-placed search box.
Be suspicious of architecture chosen before this step. A RAG pipeline built on the wrong assumption about your data is still the wrong pipeline.
Scope tools like permissions, not features
Every tool you give an agent is a capability it can misuse — through error, adversarial input, or a bad interpretation of a reasonable request. Define tools narrowly: read order status, not query the database. Create a ticket, not write to the ticketing API.
Write down, explicitly, what the agent may never do, and enforce it in code — not in the prompt. Prompts instruct; they do not restrict.
Design the failure path first
A production agent's quality is defined by what it does when it does not know. Decide upfront: when confidence is low, when a request is out of scope, or when a user is frustrated, what happens? The answer should be a designed path — a clarifying question, a human handoff with context, a graceful refusal — never an improvised guess.
Human handoff deserves real design. Transferring a conversation without its context is barely better than making the customer start over.
- Evaluation set built from real user inputs, run on every model, prompt, or pipeline change
- Tool access enumerated, least-privilege, enforced in code
- Explicit never-do list, tested with adversarial inputs
- Human handoff that carries conversation context
- Logging that lets you review any conversation end to end
- Quality sampling: a human reads a sample of conversations every week
- A kill switch and a rollback path you have actually tested
Plan for drift, not just launch
Your content changes, your products change, and model providers update their models. An agent accurate in January can quietly degrade by June. Schedule evaluation runs, review conversation samples on a cadence, and version your prompts and retrieval configuration like code — because they are.
The tradeoff to accept: every layer of evaluation, guardrails, and review adds cost and latency. The discipline is putting those layers where the risk is, not everywhere. A FAQ agent and a financial-advice agent deserve different levels of scrutiny, and your design should reflect that honestly.