Monitoring and fail-safes: a starter checklist for agentic systems

Most agent incidents don't look like a dramatic failure. They look like an agent that quietly did the wrong thing a hundred times before anyone noticed. The fix isn't a smarter agent — it's treating monitoring as infrastructure you build alongside the agent, not bolt on after the first bad week.

This is the checklist we run through before any agent we build gets access to a production system. It's deliberately scoped to what a small team can stand up in about a week, not a full observability platform rollout.

1. Instrument the whole loop, not just the model call

It's tempting to log the prompt and the response and call it observability. That misses almost everything that goes wrong in practice. An agent's real unit of work is the loop: plan, call a tool, observe the result, decide the next step. Failures usually live in that loop — a tool call that silently returned the wrong shape of data, a plan that never terminates, a decision made on stale context.

• Trace every step of the loop with a shared correlation ID, not just the final output.
• Log tool-call inputs and outputs independently of what the agent says it did — self-reported logs miss silent failures.
• Capture token counts, latency, and the specific model/version used on every span.

2. Set turn and cost budgets before you need them

An agent stuck in a reasoning loop doesn't announce itself — it just keeps calling tools and burning tokens. A hard ceiling on iterations per session (a common starting point is around 20–25 turns) forces a graceful handoff to a human instead of a silent runaway.

• Cap turns per session, and treat hitting the cap as an incident to review, not just a stopped process.
• Set a session-level cost ceiling that halts execution and pages someone before it becomes a bill you notice at the end of the month.
• Track tool_call.count and agent.steps as first-class metrics, not an afterthought in a log line.

3. Build the circuit breaker as infrastructure, not agent code

A circuit breaker that lives inside the agent's own code can fail the same way the agent fails. The pattern that holds up is a three-state breaker — closed (normal operation), open (failures exceeded a threshold, requests fail fast), and half-open (a limited test to see if the system has recovered) — implemented as a layer the agent can't reason its way around.

4. Decide what "good" looks like before launch, with evals

Evals are the difference between "the agent seems to work" and "the agent works, and we'd know within an hour if it stopped." At minimum:

1. Write 15–30 realistic test cases before launch, including edge cases you expect to be rare.
2. Run them on every change to the prompt, tools, or underlying model — not just at launch.
3. Score outcomes, not just whether the agent responded — a confident wrong answer should fail the eval.

5. Keep a human in the loop where the cost of being wrong is high

Not every action needs a human checkpoint, but the ones with irreversible or expensive consequences do — sending an email to a customer, approving a payment, closing a ticket. Draw that line explicitly before launch rather than discovering it after an incident.

An agent that fails loudly and safely is a solved problem. An agent that fails quietly is the one that ends up in a postmortem.

Where this fits in a wider program

This checklist covers the floor, not the whole practice. As agent usage grows, most teams add structured, PII-masked logging; distributed tracing with span-level model annotations; and formal SLOs for latency and error rate, the same way they would for any other production service. Our Agent Ops Console is the tool we built to cover that ground once a single checklist stops being enough.

Building or auditing an agent that's about to go into production? Get in touch — a monitoring review is often the fastest way to find out how close you actually are.