> ## Documentation Index
> Fetch the complete documentation index at: https://docs.usegradient.dev/llms.txt
> Use this file to discover all available pages before exploring further.

# Redteam testing

> Run the real production topology against a per-conversation fork of a seeded dataset instead of mocking tools, with personas drawn from the data.

A useful agent calls out to real systems. Testing it usually means one of three bad trades: run against production and book a thousand real appointments, maintain a staging copy of every downstream system and watch it drift, or test a mocked topology, which is a different agent, and the differences are exactly where bugs live.

Canned mocks look like a fourth option but are not. A tool that always returns `{"status": "confirmed"}` cannot tell you whether the agent handles a double-booking, and it has no memory. The agent books an appointment, the caller changes their mind, the agent reschedules, and the mock confirms both because it never knew about the first. Every stateful bug survives a mocked suite.

## Fork the world, keep the agent

Gradient's answer is to run the **production topology** against a forked copy of a seeded [dataset](/evaluate/datasets). Same prompts, same graph, same `http_req` rows, same model bindings, the same deployed handlers. Nothing is stubbed. What changes is the world the tools read and write: each redteam conversation gets its own writable [fork](/evaluate/datasets) of the tenant dataset, and your tool code is pointed at that fork.

## Personas come from the data

The dataset is not only the world, it is the cast. A redteam instance picks a row and impersonates it: this patient, with this chart, this insurance, this appointment history. There is no persona file to keep in sync with the fixtures, because the persona is a fixture.

That makes coverage a `SELECT`. Loop the `patients` table and each instance becomes a different caller; run 10 to 20 in parallel, each on its own fork, each impersonating someone real enough to have a history. Disposition layers on top of identity: the same row can be run cooperative, confused, or evasive.

## State survives the whole conversation

Because each conversation has a private, writable database, tool calls persist. `book_appointment` inserts a row, `reschedule_appointment` sees it, `cancel_appointment` sees the reschedule. The agent is talking to a world that remembers what it just did, the only way to test an ordinary request like "book a visit, change your mind, move it to Thursday, then cancel."

```mermaid theme={null}
sequenceDiagram
    participant RT as Redteam persona<br/>(patient 1041)
    participant A as Production topology
    participant SDK as Tools SDK
    participant F as Dataset fork<br/>(this conversation only)

    Note over F: forked from tenant dataset at conversation start
    RT->>A: "I'd like to book a follow-up"
    A->>SDK: lookup_patient(1041)
    SDK->>F: SELECT patient + appointment history
    F-->>SDK: last seen 2021-03-14
    SDK-->>A: patient, last_seen 4 years ago
    Note over A: 3-year rule, so treat as new patient
    A->>RT: asks for current insurance + address
    RT->>A: provides them
    A->>SDK: book_appointment(type: new_patient_office)
    SDK->>F: INSERT appointment
    RT->>A: "actually, can we move it to Thursday?"
    A->>SDK: reschedule_appointment(...)
    SDK->>F: UPDATE, sees the row it just wrote
    F-->>SDK: rescheduled
    A-->>RT: confirms Thursday
    Note over F: fork discarded, tenant dataset untouched
```

The rules that grade this run read both sides: the transcript (did it ask for new insurance?) and the fork's final state (is the row `new_patient_office` rather than `follow_up`?). See [rubrics](/evaluate/rubrics).

## One handler, no mock branch

Your [custom tool](/build/custom-tools) resolves which database it is talking to from authenticated redteam context, so the same code serves production traffic and redteam traffic:

```js theme={null}
export const bookAppointment = tool({
  name: "book_appointment",
  // `db` is the tenant database for real traffic, this conversation's
  // fork for redteam traffic. Same query either way.
  async handler({ patientId, slot, type }, { db }) {
    return await db.appointments.insert({ patientId, slot, type })
  },
})
```

Under the hood, the fork's connection string is provided to the tool at runtime, and the SDK's `db` helper uses it automatically. This is the payoff of forking over stubbing: there is no second implementation to drift, because the code under test is the code that ships.

## Tools that leave the dataset keep a mock

A fork can contain your dataset, not the outside world. Tools that reach past it (charging a card, sending an SMS) still need an explicit `mock` handler.

<Warning>
  A redteam can never reach a real external system. Redteam context with no mock defined fails loudly rather than falling through to the real call. And because that context is authenticated, not a client-settable flag, production traffic can never be routed to a fork.
</Warning>

## What a run produces

Each conversation produces a [trace](/evaluate/results) (every turn, every swap, every tool call with its arguments and response) plus the final state of its fork. Both feed [evaluation](/evaluate/rubrics).

<CardGroup cols={2}>
  <Card title="Datasets" icon="database" href="/evaluate/datasets">
    The seeded world a redteam forks, and the rows a persona comes from.
  </Card>

  <Card title="Custom tools" icon="screwdriver-wrench" href="/build/custom-tools">
    Write one handler that serves production and redteam alike.
  </Card>
</CardGroup>