Agent engineering / Field guide

What actually makes an AI agent production-ready?

A prototype demonstrates that a model can do something once. Production engineering is the work required to make that behavior dependable, accountable and recoverable.

The first constraint is tool access

An agent is useful when it can take action through tools: fetch a record, update a ticket, query a database, or draft a proposed change. Those actions also create operational risk. Each tool should enforce authorization independently of the model's instructions.

Prefer least-privilege credentials, allowlisted operations, explicit input validation, and human confirmation for irreversible or financially meaningful actions.

Evaluation belongs in the build loop

It is difficult to improve a system you cannot measure. Build a representative evaluation set before optimizing prompts or switching models. Include ordinary cases, ambiguous cases, adversarial inputs and cases where the correct behavior is to refuse or escalate.

Measure task completion and correctness, but also the rate of unsafe or unwanted actions. Record the version of the prompts, tools and policies used for each evaluation run.

Observability changes debugging

A standard request log is insufficient for multi-step agents. Capture the sequence of model decisions, tool invocations, permissions, external dependencies, latency, token use, and human approvals. Apply appropriate data minimization and redaction to traces.

Design for controlled failure

Tools time out. Models produce malformed output. Documents contain conflicting instructions. Retries can repeat side effects. Use timeouts, bounded retries, idempotency keys where appropriate, schema checks and explicit fallback states.

Successful demos demonstrate capability. Reliable systems demonstrate control.

A practical readiness checklist

• Every side-effecting tool has independent access control.
• Critical actions have confirmation, review or rollback.
• Evaluation covers normal, ambiguous and failure cases.
• Traces are available without leaking confidential material.
• Operating cost, latency and failure recovery are defined.
• Someone owns the runbook and change process after deployment.

If you already have a prototype, a focused production-readiness review can identify the highest-risk gaps before a larger deployment commitment.