State, Tools and Approval Gates: A Checklist for Safe Agents

Deployment of agentic systems in enterprise environments fails when safety is treated as a prompt-level instruction rather than an architectural constraint.

In Tuesday’s core article, Governed Agentic Workflows: Architecture for Enterprise AI, we established that governance is not a policy document—it is a set of rigid boundaries (tool scope, state audit, and approval gates) baked into the system.

Today’s field note provides the implementation counterpart: a practical checklist for engineering leads and architects to use during design reviews and pre-production gates. This is derived from patterns used in Agent Shaker and refined through production delivery experience.

The Architectural Safety Shift

Most "safe agent" approaches rely on system prompts like "Do not delete data unless the user asks." This is insufficient. A safe agent architecture assumes the LLM will deviate, and ensures the surrounding system prevents that deviation from causing damage.

We categorise these constraints into three domains: Tools, State, and Approval.

1. Tool Governance Checklist

Tools are the agent's only mechanism for affecting the world. If tools are not scoped, the agent is inherently unsafe.

• Granular Tool Scoping: Are tools provided as monolithic APIs or discrete, single-purpose functions? (e.g., UpdateRecordStatus instead of a generic ExecuteSQL).
• Read-Only Defaults: Are all tools read-only by default, requiring explicit elevation for write actions?
• Execution Budgeting: Is there a hard limit on the number of tool cycles per task to prevent infinite loops or excessive cost?
• Input Sanitization: Does the tool implementation validate agent-generated inputs against a strict schema (e.g., Pydantic or JSON Schema) before execution?
• Destructive Action Rollback: For tools with side effects, is there a defined "undo" or compensating transaction, or is a human-in-the-loop gate mandatory?
• Identity Propagation: Does the tool execute under the agent’s permission context or the end-user’s? (Avoid "God-mode" service accounts).

2. State & Audit Checklist

An agent without a verifiable state is a black box. You cannot govern what you cannot reconstruct.

• Immutable Task Log: Every tool call, prompt, and LLM response must be logged to an append-only store with a timestamp and parent task ID.
• Context Window Hygiene: Is the context provided to the agent limited to the current task state, or is there "noise" leakage from unrelated operations?
• Snapshotting: Can the system reconstruct the exact state of the agent at any point in the execution timeline for post-incident analysis?
• State Reconciliation: In multi-agent flows, is there a single "source of truth" task state, or is state fragmented across agent memories?
• Data Retention Compliance: Does the state log automatically purge or mask PII/Sensitive data to meet regulatory obligations?

3. Approval Gate Checklist

Approval is not a checkbox; it is a first-class state transition in the workflow.

• Explicit Uncertainty Triggers: Does the agent have a defined "Escalate to Human" tool for when confidence scores fall below a threshold or ambiguity is detected?
• Dynamic Gate Placement: Are approval gates placed before execution of high-risk tools, or only at the end of a long chain? (The former is mandatory for safety).
• Full-Context Reviews: Does the human approver see the final requested action, the rationale (CoT), and the data source used to justify the action?
• Bypass Prevention: Is the workflow engine hard-coded to refuse execution of certain tool-sets unless a cryptographically signed approval token is present?
• Escalation SLAs: For interactive systems, what is the fallback if an approval gate is not answered within the required window?

The "Agent-Ready" Gate

In a typical delivery cycle, this checklist should be applied twice.

First, during the Architecture Decision Record (ADR) phase, to ensure the infrastructure supports these constraints. Second, during the Readiness Review, where the specific agent prompts and tool definitions are tested against these rules.

We hit the clean version of this. An agent reasoning through a multi-step task referenced a tool that was not in the registry — a plausible-sounding name that did not exist. The gate caught it before the call went out.

What matters is where the check sits. Nothing about that call was malicious, and nothing about it was a reasoning failure you could fix with a better prompt. The registry is authoritative: a tool that does not resolve is not a tool, and resolution is deterministic. No amount of model confidence changes that.

This is the cheapest control in the whole design, and the one most often skipped — resolve the tool name before you plan the arguments. A model inventing a function name is ordinary. A model inventing a function name and then executing it against your production system is an incident report.

Artifact: The 3-Domain Safety Matrix

Related: Governed Agentic Workflows: Architecture for Enterprise AI | When Should You Use an AI Agent Instead of a Workflow? | Project: Agent Shaker

Sources

• MCP Specification, revision 2026-07-28 — Tools — human in the loop, confirmation prompts, untrusted tool annotations