How to Read an AI Agent Trace and Find the Root Cause (2026)
A trace is the whole story of one agent run. How to read one, what a waterfall shows, and the patterns that point straight at the cause.
On this page
Logs tell you that something happened. A trace tells you the story: what the agent was asked, what it decided, which tools it called in what order, how long each step took, what it cost and where it went wrong. When an agent misbehaves, the trace is the first thing to open, and knowing how to read one turns an hour of guessing into a few minutes.
This guide explains what a trace contains, how to read it, and the five patterns that point straight at a root cause.
What a trace is (and what it is not)
A trace is all the work an agent did for one task: one question answered, one ticket handled, one coding request completed. It is made of events (a model call, a tool call, a decision, an error), each with a time, a duration and an outcome.
Two other groupings are easy to confuse with it:
- A session is everything that happened in one process lifetime, for example one Claude Code launch. A session contains many traces.
- A span is one timed step inside a trace. Sources that speak OpenTelemetry (the Vercel AI SDK, Codex, Python apps) send spans with parent-child links, so a trace can be drawn as a tree.
If your traces are too long to read, they are probably sessions. If they are too short to explain anything, the trace id is being reset in the middle of a task.
Read the waterfall first
The waterfall puts every step on a shared timeline: one row per step, a bar for how long it ran, nested under the step that triggered it. Before reading any content, it already answers three questions:
- Where did the time go? Look for the longest bars. A single slow tool call often explains a "slow agent".
- What ran in parallel and what waited? Gaps between bars are time spent waiting, usually on the model.
- Where did it fail? Failed steps stand out; the first one matters more than the last.
Agents that send flat events (hooks from Claude Code or Cursor, wrapped OpenAI calls) still get a waterfall: each event becomes a lane at its own time, tool calls as bars and instant events as ticks.
The five patterns that point at the cause
| Pattern | What it looks like | What it usually means |
|---|---|---|
| First failure | The earliest failed step, some seconds into the run | The root cause. Later failures are usually consequences |
| Latency hotspot | One step takes 40% or more of the whole trace | A slow tool or API, a missing timeout, a huge payload |
| Cost hotspot | One step carries 60% or more of the cost | An oversized context, a retry with the full history, the wrong model |
| Retry loop | Three or more identical calls in a row | The agent is stuck: a tool keeps failing the same way, or no stop condition |
| Off baseline | Duration or cost far from this agent's usual runs | Something changed: input size, model, a dependency |
Read them in that order. The first failure is the single most useful fact in a broken trace: the step where reality and the agent's plan first disagreed.
Read the content around the first failure
Once you know the step, read three things side by side:
- What the agent asked for: the tool name and its arguments.
- What came back: the raw output or error.
- What the agent did next: the following decision.
Root causes live in the gap between 2 and 3: an error message the model treated as data, an empty result read as "nothing to do", an answer from the wrong record. If the content is not there, the trace was captured at a privacy level that drops it; timings and tool names still narrow the cause down.
Compare against a healthy run
A single trace shows what happened; a pair shows what was different. Open a successful run of the same task and compare step counts, durations and tool order. Where the two paths split, check the input: a longer context, another file, a newer model, a slower dependency.
How Smart Traces do this for you
Voight's Smart Traces apply this method to every trace automatically:
- Waterfall for every source, with a tree for OpenTelemetry spans and timed lanes for flat events. Each step shows its duration, tokens and cost.
- Smart analysis. A deterministic diagnosis runs on every trace and reports the first failure with its offset, latency and cost hotspots, retry loops of three or more identical calls, and how the run compares with the agent's median over the last seven days. When nothing is wrong, it says so.
- Explain with AI. One click sends the findings and a trimmed timeline to a model and returns a plain-language explanation of what happened. The result is cached per trace and can be regenerated.
- Issues only. A toggle in the trace list hides healthy runs so you can go straight to the broken ones.
- Every source. Claude Code and Cursor hooks, Codex over OpenTelemetry, wrapped OpenAI and Anthropic clients, the Vercel AI SDK, Python apps and the plain HTTP API all land as traces.
Connect a coding agent with one command:
npx -y @voightxyz/sdk setup
Checklist
- One trace per task, not per session and not per model call.
- Every step has a start time, a duration and an outcome.
- Read the waterfall before the content.
- Start from the first failure, not the last.
- Check the hotspots and loops before reading prompts.
- Compare with a healthy run of the same task.