Read a graded run by scenario and by node, and use native agent traces (transcript, spans, and rubric grade) to turn a failing rule into a fix with an address.
Every conversation Gradient runs (a redteam fork, a dashboard chat, a phone call, an API turn) is recorded as a trace, automatically. There is nothing to instrument. A trace is where a graded run stops being a number and becomes something you can act on.
Each conversation produces a transcript (every turn, every node swap, every tool call with its arguments and response) plus, for a redteam, the final state of its fork. Both feed evaluation: the transcript answers “what did it say,” the tool calls and fork state answer “what did it do.”A trace opens on three tabs.
Transcript
Spans
Rubric
The conversation as it happened: caller and agent lines, each tool call with its input, output, duration, and status, and the conversation variables a turn changed. Every line is attributed to the node it ran under. For voice, the transcript also shows what the caller actually heard: the greeting, holding phrases, and backchannels the engine played.
The waterfall beneath the dialogue: each turn, the model calls inside it, and the tool calls beneath those, with duration, tokens, and status. Expand any span for its raw input, output, and attributes: the view to reach for when a single turn misbehaved.
The grade, rule by rule. Whole-conversation rules first, then a group per node, each rule a pass, fail, or skip badge with the judge’s reason. Grade this trace runs the rubric on demand; earlier grades stay as history.
The header carries the facts you scan for first: which agent and published version served the call, its duration, the tool-call count and token total, whether it is still ongoing, and, for voice, whether the agent or the caller hung up.
Agent traces are kept for a rolling window of 14 days. Copy the rendered transcript or the raw spans straight from a trace to paste into an issue.
A single trace tells you what one conversation did. A run’s value is in the aggregate, and it rolls up two ways.
By scenario
Where to spend the next iteration. “Follow-up visits pass 61%” names a definition of correct the agent is not yet meeting.
By node
Which prompt to edit. “patient_lookup failed 4 rules” names one node in the graph.
The node axis is what closes the loop. A failing rule is not a vague quality complaint; it is a work item with an address. It names the node; you open that node in the builder, fix the prompt, publish a new version, and the next run tells you whether the number moved.Because the same rubric grades production and redteam, this is one loop, not two. A regression the redteam surfaces before you ship and a regression a real caller hits after you ship land on the same scenario and the same node, and get fixed the same way.
Rubrics, scenarios, and rules
The definition of correct that every grade is measured against.
Agents
Open the failing node in the builder and edit the prompt.
