AI agent observability: what to trace, what to measure and when to alert

Why logs are not enough

An ordinary service runs the same code on every request, and a log line with latency and status code describes it almost completely. An agent may make two tool calls or seven for the same question, take a different route, loop, or stop early. A record saying “answered in 4 seconds, status 200” says nothing about that request.

What you need is a trace: the tree of steps in one run, where every model call, tool call and knowledge-base lookup is a node with its own input, output, latency and cost. Without it every user complaint becomes “cannot reproduce” and every model change becomes a lottery.

What to record from day one

The minimum set is small, and it is cheaper to collect from the start than to bolt on after the first incident.

• Per run: trace id, user or tenant, prompt version, model and model version, total tokens, cost, duration, outcome: success, refusal, error, or stopped by a limit.
• Per step: node type (model call, tool, retrieval, human check), tool arguments and result, input and output tokens, latency, error if any. Prompt and completion content is stored separately, with a switch to disable capture for sensitive data.
• Per decision: which route the agent chose and why. If the system has a “light model or top-tier model” router, its decision belongs in the trace, otherwise you can explain neither the bill nor the quality.
• Links to the outside world: the request id in your backend and the id in the model provider's system. When the provider returns an error you need their id, so that your support ticket does not read “it fails sometimes”.

Four signals that deserve alerts

Nobody looks at a dashboard with forty charts. Alerts belong on four quantities, and each threshold is set at launch, not after an incident.

• Cost per task. Not the daily total but the p95 cost of a single run. A rising p95 at constant load means the agent started taking more steps, and that shows within hours, not at month end.
• p95 latency. The average hides the tail: the ten percent of requests that wait half a minute are the ones people write about.
• Tool error rate. An empty CRM response or a knowledge-base timeout after which the agent kept reasoning as if it had the data. An empty result must be an explicit error in the trace, and a rising share of them is the signal.
• Step count and limit stops. Every task has a limit on steps, time and cost. A triggered limit means a loop or a degraded tool, and it needs to be visible separately from ordinary errors.

Quality is measured separately

Traces and metrics answer “what did the agent do”. They do not answer “was it right”, and no observability tool answers that on its own. For that you need a set of examples with known correct outcomes, run on every change to the prompt, the model or a tool.

How it works for us. For the safety layer of one of our products the set has 30 scenarios: 12 where the model must fire, 8 where it must stay silent, 10 borderline. Each scenario runs five times, 150 calls per model. The criteria are fixed in advance: zero misses, precision at least 0.92, valid JSON 150 out of 150, p50 under 2 seconds, p95 under 3.5 seconds, price at most one dollar per thousand calls. The trade-off order is fixed too: recall, schema validity, precision, p95, price.

Production traces feed this set. Every incident dissected through a trace becomes a new scenario. That way the set grows from real failures rather than from guesses.

Where the standard stands: OpenTelemetry GenAI

OpenTelemetry has semantic conventions for GenAI: shared attribute names for model calls, tokens, agent steps, tool and MCP calls, content events and evaluation results. They exist so that the same instrumentation code can be sent to Langfuse, Phoenix or your own database without a rewrite.

An important caveat as of October 2026: the conventions are not stable. In June 2026 they moved out of the main repository into a dedicated one, open-telemetry/semantic-conventions-genai, and on the check date the specification documents carry Development status with no tagged release. In practice: pin the convention version in code, expect attribute renames, and do not build dashboards on names from last year's blog posts.

Tools: which one fits which project

There are more than a dozen tools in this category. For projects of our scale the choice usually comes down to three options plus “own backend on top of OpenTelemetry”. Licence facts checked on 29 October 2026 against the repository pages.

A minimal setup for the first week

If the agent is already in production and there is no observability, this is the order.

• Wrap every model call and every tool in a span with tokens, latency and result. One day of work if the tools are already described as functions.
• Add per-task limits: steps, time, cost. A triggered limit is written to the trace as a distinct outcome.
• Stand up Langfuse or Phoenix next to the application and send traces there. One instance without high availability is enough at this stage.
• Set the four alerts from the section above, with thresholds at current p95 values plus headroom.
• Build the first 30 evaluation scenarios from last week's traces: ten successful, ten questionable, ten failed.