Insights · · 2 min read
A practical blueprint for governed AI agents
An agent that can only talk is a chatbot. An agent that can act is a coworker, and you would not give a new coworker admin credentials, no supervision, and no record of what they did. The blueprint below is how we ship agents that operations teams actually trust.
Start from the action, not the model
Design begins with the last step: what will the agent change in the world? A corrected order, a drafted customer email, a reprioritized ticket. Enumerate those actions, rank them by blast radius, and decide per action whether the agent may execute, may only draft, or may only recommend. Everything upstream, retrieval, reasoning, and tools, is then built to justify that one action.
Principle 1: scoped access, inherited from your systems
Agents get credentials the way employees do: least privilege, per task, revocable. Retrieval respects the permissions of the person who triggered the run, so an agent asked by a sales rep cannot read finance data the rep could not open. No pooled super-user connections, ever. This one rule removes the largest class of AI incidents before they can happen.
Principle 2: evidence on every step
Every run produces a trail: what was retrieved and from where, which tools were called with which inputs, what the model was asked, and what it produced. Answers carry citations to source records. When the agent proposes an action, the proposal packages its evidence, so the approver sees the why, not just the what.
Principle 3: drafts by default, approval for impact
The agent drafts; a named person approves anything that writes to a system of record, sends an external message, or moves money. Approval gates are defined per action type, with escalation and deadlines so the queue cannot silently rot. Reversible actions may earn automation over time; irreversible ones keep the gate. The approval history, including the evidence shown, is retained.
Principle 4: routing instead of loyalty
Model choice is a policy, not an architecture. A gateway routes each call by data sensitivity, capability, and cost: a commercial API with training opt-outs for general drafting, a self-hosted model for the workloads that must not leave your accounts. When a better or cheaper model ships, the policy changes and the system stays.
Principle 5: evaluations before autonomy
Trust is earned with numbers. Before an agent touches production, it runs a gauntlet of real cases with known answers. After launch, the same suite reruns on every prompt, model, or policy change, plus a sample of live traffic. Autonomy expands only where the scores justify it, and shrinks where they do not.
The minimum viable governed agent
- One workflow with a named owner and one instrumented metric
- Scoped credentials and permission-aware retrieval
- Citations and full run logs
- Draft-plus-approval for every impactful action
- A model gateway, even with one model behind it, so routing is possible later
- An evaluation set that grows with every mistake
Teams sometimes read this list as overhead. It is the opposite: the governance is what lets the agent do more. An agent that can prove what it did gets given real work. One that cannot stays a demo.
Frequently asked questions
- Do human approval gates defeat the purpose of AI agents?
- No. The agent still does the detection, investigation, evidence gathering, and drafting, which is most of the work. The approver spends seconds on a decision that used to take an hour of digging, and the business keeps accountability.
- When is it safe to let an agent act without approval?
- When the action is reversible, the evaluation scores for that action type have been consistently strong, and the run still logs full evidence. Expand autonomy per action type, based on measured performance, never as a global switch.
- What is the biggest mistake teams make with AI agents?
- Giving the agent a pooled, over-privileged credential to make integration easy. It turns every prompt into a potential permissions bypass and makes incidents impossible to attribute. Scoped, per-task credentials are non-negotiable.