Take an agent from demo to production

Use this when

You have a working prototype and people are asking when it can go live.

What you need

• A working prototype, and its spec
• 30 real past cases
• Someone with access to set permissions
• A named owner

The steps

1. 1
   Write down what it's allowed to do

   List every tool and every action. Remove anything the first version doesn't need.

2. 2
   Enforce limits in code

   Permissions, spending caps and step limits belong in your system, where a prompt can't talk its way around them.

3. 3
   Add approval for risky actions

   Anything customer-facing or hard to undo waits for a person.

4. 4
   Log every run

   Record inputs, tool calls, outputs and cost, so you can replay any failure.

5. 5
   Build the test set

   Use 30 or more real cases, including attempts to mislead it through the content it reads.

6. 6
   Run in shadow mode

   Let it work on live cases without acting. Compare its choices with your team's.

7. 7
   Release to a small group

   Start with one team or low-risk cases. Agree in advance what would make you pull it back.

8. 8
   Assign an owner

   Set failure alerts, a manual fallback, and a regular review of quality and cost.

A prompt to start from

Edit the parts in brackets, and test it on your own cases.

Review this run log as a sceptical reviewer.

Goal of the agent: [goal]
Rules it must follow: [rules]
Log: [paste tool calls and outputs]

List:
1. Any step that broke a rule
2. Any step that used a tool it didn't need
3. Any claim in the final output not supported by the log
4. What should have triggered a hand-off to a person
If you find nothing in a category, write "none found".

How you'll know it's working

Decide your pass mark before you test. Measure how accurate your team is today on the same cases, and use that as the number to beat.

• No high-severity failures in the test set
• Shadow mode results match your team's on the cases that matter
• You can explain any run from its log
• Turning it off takes minutes, and the manual fallback works

Where this goes wrong

• Going live because the demo went well
• Enforcing limits only in the prompt
• Changing the model without rerunning the tests