How to Evaluate Enterprise AI Agents Before Production

An AI agent can produce a convincing answer while failing the job. It can also pass a release review, change underneath the workflow, and keep carrying an approval that no longer describes the system in production.

Recent releases make that approval gap concrete.

OpenAI released GPT-6 Astra for work that includes computer use, software engineering, and other tool-driven tasks, and described it as the company’s first model to reach the Critical level of cybersecurity capability under its Preparedness Framework.[1][2]

Anthropic released Claude Fable 5.1 and Claude Mythos 5.1 as the same model with different safeguard and access regimes.[3] Its Enterprise Frontier Safeguards design stores activity in customer-controlled cloud infrastructure and can correlate signals across sessions and accounts.[5]

Google released Gemini 3.8 Flash for long-horizon software engineering, autonomous agents, and complex enterprise workflows.[4]

Those are vendor statements, not proof that any model is ready for a specific enterprise workflow. They do show why model identity, access, monitoring, and safeguards belong inside the production approval record.

A model upgrade creates a new agent version. It does not inherit the old version’s approval.

Define the version you are approving

An agent is a composite system. The model matters, but so do the instructions, tools, identity, permissions, knowledge sources, approval rules, runtime, and operating environment around it.

Record the complete release identity before testing:

• model provider, model name, and snapshot;
• reasoning or effort setting;
• system instructions and policy version;
• available tools and tool schemas;
• service identity and permissions;
• retrieval sources and knowledge versions;
• approval and escalation rules;
• runtime, region, and deployment configuration;
• evaluation-set version, release owner, and date.

This record answers a basic incident question: what exactly was running when the decision was made? It also stops teams from treating a model alias as a stable component when the approved behavior depended on a particular snapshot and configuration.

Version the composite system, not only the prompt.

Freeze the operating contract before comparing models

Do not begin with benchmark scores. Begin with the job.

Write the operating contract in terms the workflow owner can inspect:

• What triggers the agent?
• Which business outcome must it produce?
• Which data may it read?
• Which tools and actions may it use?
• Which actions need human approval?
• Which actions are prohibited?
• What counts as completion, escalation, refusal, or failure?
• Which latency and cost limits apply?
• Who owns the outcome after release?

The contract should stay fixed while the old and new versions run through the same work. If the task, permissions, rubric, and model all change at once, the comparison cannot tell the release committee what caused the result.

If the organization has not yet defined the workflow owner, baseline, authority boundary, or release path, use an AI readiness assessment before selecting a model.

Decide which evidence the change invalidates

A change does not always require rebuilding every test. It does require a written decision about which prior evidence still applies.

Write this as an evidence-impact note attached to the release. A one-line “model upgraded” ticket is not enough for a system that can read records, call tools, or change external state.

Run both versions on the same work distribution

A polished demo hides variance. A production decision needs repeated runs across the work the agent will actually encounter.

Use normal cases from major workflow categories, then add the conditions that usually break orchestration: incomplete inputs, conflicting records, stale documents, unavailable tools, permission failures, duplicate events, delayed approvals, requests outside scope, prompt injection, and tasks with no valid answer.

Keep a held-out set for the release decision. Include known failures from incidents and near misses. Run enough repetitions to expose unstable tool selection or stopping behavior instead of accepting the best attempt.

Compare the current production version with the proposed version on the same cases. Review differences by case class. An aggregate average can hide a severe authority failure inside a broad improvement in writing quality.

Inspect the trajectory and the final state

The final response is only one part of the run. Capture the sequence that produced it:

1. objective recognized;
2. evidence gathered;
3. tool selected;
4. arguments submitted;
5. state changed;
6. retry or recovery attempted;
7. approval requested or bypassed;
8. escalation, refusal, or completion recorded;
9. final external state verified.

A correct answer reached through an unauthorized action is a failed run. A safe trajectory that never finishes within the service requirement also fails. The review needs both conclusions because they lead to different fixes.

Use deterministic checks for permissions, required fields, calculations, state transitions, duplicate prevention, protected actions, and final system state. Reserve judgment rubrics for qualities that need judgment, such as usefulness or whether an escalation contains enough context for a human decision.

Re-test authority, recovery, and monitoring

Greater capability can change which instructions the agent follows, which tools it selects, and how far it proceeds before asking for help. That makes authority testing part of the release suite.

Verify that the proposed version:

• starts with the minimum access required for the job;
• cannot expand its own permissions;
• ignores instructions embedded in untrusted documents or tool output;
• cannot cross tenant, account, or record boundaries;
• asks for approval before a protected action;
• shows the approver the action, evidence, and consequence;
• fails closed when identity, policy, or approval is unavailable;
• leaves enough evidence to reconstruct what happened.

Then inject operational failures. Test duplicate triggers, partial tool results, lost responses after a successful state change, unavailable dependencies, restarts, late approvals, and human corrections that arrive during a retry. The system must distinguish an action that is safe to repeat from one that requires reconciliation.

Separate provider safeguards from controls the enterprise owns. Anthropic’s Fable and Mythos release shows that closely related access paths can carry different safeguards.[3] Its Enterprise Frontier Safeguards design also makes a specific architectural tradeoff: activity stays in customer-controlled infrastructure while automated monitoring can correlate behavior across time and accounts.[5]

Record those provider properties, but do not substitute them for workflow tests. The enterprise still owns identity, permissions, approval design, data boundaries, incident response, and the final release decision.

Price the accepted outcome

Token price does not tell you what a production workflow costs.

Measure model charges, tool calls, retries, reviewer time, failed actions, recovery work, and infrastructure. Then divide the total by accepted outcomes, not attempted runs. A cheaper call can become an expensive workflow if it retries more often or creates more review work. A more capable model can lose the comparison if its behavior is harder to control on the bounded job.

Track quality, latency, and cost together. Keep the dimensions visible. The release committee should be able to see what improved, what regressed, and which tradeoff it is accepting.

Make re-approval explicit

End the review with one of four decisions:

• Promote: the new version meets the existing contract and release criteria.
• Constrain: release it to a smaller cohort, narrower tool set, lower authority, or explicit approval path.
• Continue evaluation: a material unknown has a defined test and owner.
• Roll back: keep the prior version because the new one weakens safety, reliability, operations, or economics.

Retain the previous model snapshot until rollback has been tested. Version the evaluation set and acceptance criteria with the release. Record who approved the decision across Engineering, Security, Product, Operations, and the accountable executive.

The decision record should answer five questions without requiring a new investigation:

1. What changed?
2. Which prior evidence became invalid?
3. What did the side-by-side evaluation show?
4. Which risks remain, and how are they bounded?
5. Who owns the next review or rollback?

After release, sample production runs, capture operator corrections, classify incidents and near misses, and add each novel failure to the regression set. New customers, policies, documents, tools, and workflow variants can invalidate the original sample. Teams using a governed knowledge layer such as CortexSeed should version that context alongside the agent and evaluation set.

An enterprise agent is ready when the organization can defend this version of the whole system for this job, this authority, and this operating environment. A new model name does not change that standard.

VallySeed’s Enterprise AI Agent Development engagements connect workflow contracts, tool integration, evaluation, authority boundaries, and production operations. If a pilot still lacks an owner, baseline, or release path, start with the guide to moving an AI pilot into production.

Review the governance boundary. Bring one current agent workflow, its authority map, and the acceptance criteria used for the last release. We will identify which evidence still applies and which production gates the proposed change must re-open.

Sources

[1] https://openai.com/index/gpt-6-astra — GPT-6 Astra: A new generation of intelligence [2] https://openai.com/index/safety-overview-gpt-6-astra — Safety overview: GPT-6 Astra [3] https://www.anthropic.com/claude-fable-and-mythos-5-1 — Introducing Claude Fable 5.1 and Claude Mythos 5.1 [4] https://ai.google.dev/gemini-api/docs/changelog — Gemini API release notes [5] https://www.anthropic.com/news/enterprise-frontier-safeguards — Developing Enterprise Frontier Safeguards with our customers