A 90% AI agent pass rate describes one trial. If ten independent runs must all succeed, that rate gives only a 34.9% chance of a clean ten-run sequence. Over twenty runs, the figure falls to 12.2%.
The release decision therefore needs a reliability metric shaped like the work. A research task with cheap retries can reward one successful result across several attempts. A daily workflow that updates customer, financial or operating systems needs consistency, bounded recovery and a separate gate for consequential errors.
Repetition changes the reliability number
A per-run pass rate describes one trial sampled from the evaluation set. Recurring operations expose the system to that probability again and again.
Under a simple independence assumption, the chance that every run in a sequence succeeds is:
sequence reliability = per-run pass rate ^ number of required runs
The assumption will not hold perfectly in production. Shared outages, stale data and prompt changes create correlated failures. The calculation still gives a useful first test because it reveals what the headline average leaves out.
| Per-run pass rate | Five clean runs | Ten clean runs | Twenty clean runs |
|---|---|---|---|
| 80% | 32.8% | 10.7% | 1.2% |
| 90% | 59.0% | 34.9% | 12.2% |
| 95% | 77.4% | 59.9% | 35.8% |
| 99% | 95.1% | 90.4% | 81.8% |
These figures test whether the unit of measurement matches the operating promise. A team approving a daily planning artefact, for example, needs to know how frequently the sequence breaks, how much review repairs it and whether the failed run delays a decision.
Anthropic’s guide to evaluations for AI agents defines a task as one test case and a trial as one attempt. It recommends multiple trials because model outputs vary. That distinction matters: a task can be solvable while the workflow remains too inconsistent for the intended cadence.
Choose the metric around the retry model
Two metrics can use the same trials and support opposite decisions.
Pass@k asks whether at least one attempt succeeds within k trials. It suits capability discovery and workflows where attempts are cheap, independent verification is reliable and retries create no harmful side effects. Code generation against a deterministic test suite is a plausible example: several isolated attempts can run, and the system accepts only a result that passes the tests.
Pass^k asks whether all k trials succeed. It tests consistency. This view fits scheduled reports, repeated case handling and action-taking workflows where every failed attempt creates an exception, delay or recovery burden.
Suppose an agent has a 70% chance of success on one attempt. With three independent attempts, the chance of at least one success is 97.3%. The chance that all three succeed is 34.3%. Both numbers are correct, but only one reflects a workflow that requires three clean cases.
Retries also change the economics. Three attempts can rescue capability while tripling model and tool usage, adding latency and increasing the number of traces a reviewer may need to inspect. When an attempt changes external state, parallel retry is dangerous unless the action is isolated, idempotent and reconciled against the destination.
Pass@k fits systems that can generate candidates safely and verify the winner. Sequence reliability fits work where each case must complete cleanly. Report both when the workflow retries, because the gap between them is the operating cost hidden behind the capability score.
The pass condition must inspect the outcome
A reliable metric can still grade the wrong object. An agent may produce a polished confirmation while the destination system remains unchanged.
Anthropic separates the transcript from the outcome: the trace records the agent’s steps, while the outcome is the final state in the environment. Microsoft’s agent evaluators make a related distinction between system evaluation and process evaluation. Task completion and adherence assess the overall result; tool selection, input accuracy, output use and call success inspect the route taken.
Build the pass condition in three layers:
- Check whether the CRM, repository, ticketing system or approved artefact reached the required business state.
- Verify that the run stayed within its instructions, permissions and approval rules.
- Inspect whether the agent called the correct tools with valid parameters and used their responses accurately.
A run passes only when the layers required by that workflow clear their gates. Successful text cannot compensate for a missing database update. A correct final state does not excuse an unauthorised retrieval or external action.
OpenAI’s trace grading documentation supports structured labels across the end-to-end record of decisions and tool calls. Its Agents SDK captures model turns, function calls, handoffs and guardrails, giving evaluators enough evidence to identify where the result diverged. Trace data can contain sensitive inputs and tool payloads, so capture and retention need the same access discipline as the underlying workflow.
Segment the rate before it reaches the release meeting
An aggregate pass rate blends easy cases with expensive failures. If routine work dominates the sample, the average can improve while performance on the pressured segment remains flat.
Split results by the conditions that change consequence or mechanism:
- case type and customer segment;
- source availability and freshness;
- tool path and destination system;
- autonomous versus approval-gated action;
- normal work, known exception and adversarial case;
- first attempt, retry and recovered run.
Consider 100 test cases where 95 routine cases pass and all five high-value exceptions fail. The dashboard shows 95%. The release owner sees an agent that breaks precisely where judgement and customer risk concentrate.
Give consequential failure classes their own hard gates. Approval bypass, restricted retrieval, incorrect payment, unsupported customer commitment and silent write failure should never be averaged away by fluent low-risk outputs. This is a policy decision expressed as an evaluation rule.
For teams building in Microsoft’s platform, the Copilot Studio agent evaluation guide maps those gates to test sets, evaluation methods, user profiles and case-level release evidence.
The same segmentation should reach cost. Record model and tool spend, active review, exception handling and recovery for each accepted outcome. The AI agent unit economics framework explains why initiated runs provide a weak denominator when the business rejects part of the output.
Convert the metric into a release rule
A production gate needs more than a target percentage. Write the workload, measurement window, required sequence, retry policy and consequence limits beside the number.
A useful release record contains:
| Field | Decision it forces |
|---|---|
| Unit of work | What exactly counts as one trial? |
| Eligible segment | Which cases does the rate describe? |
| Outcome verifier | Which system or owner confirms success? |
| Trials per task | How much model variation was tested? |
| Required sequence | How many clean cases must the operation sustain? |
| Retry budget | How many attempts, at what cost and latency? |
| Hard failure gates | Which events stop release regardless of the average? |
| Review load | How much human time turns a generated result into an accepted one? |
| Recovery rule | What happens after partial or uncertain execution? |
| Re-test trigger | Which model, prompt, tool, source or policy changes reopen the gate? |
Use capability evals to find the edge of what the agent can do. Anthropic distinguishes these from regression evals, which protect tasks the system already handles and should approach a near-perfect pass rate. Promotion happens when the representative regression suite, hard consequence gates and workload-level reliability all support the same boundary.
The sample size also belongs in the decision. Nine passes from ten trials produces a preliminary estimate with wide uncertainty. Expand the trial set across realistic variation, report the count beside every percentage and preserve failures as regression cases. The operational target becomes harder to game because reviewers can see both coverage and uncertainty.
For broader pilot design, use the AI agent pilot success criteria guide. When a failed case needs causal diagnosis, the six-layer troubleshooting method connects the outcome back to context, judgement, tools, authority and handoffs.
Measure the operating promise
An AI agent reliability metric earns trust when it predicts the workload buyers and operators will experience. That requires a denominator tied to real cases, a pass condition tied to verified state and a sequence length tied to the operating cadence.
Model Operator applies this discipline to recurring, evidence-backed workflows. The current design-partner wedge centres on one Product × GTM Planning Room artefact, measured through acceptance, revision, provenance and repeat use while approval bypass and restricted retrieval remain hard-stop events. The team is implementing and validating that operating layer; production reliability depends on accepted outcomes across repeated work.
If your agent has an impressive pass rate but no workload-shaped release rule, bring one recurring workflow to Model Operator or email alexander@modeloperator.io. We will define the outcome, trials, consequence gates and evidence required before authority expands.