The S.H.I.P. playbook: taking an agent design to production
Scope, Harden, Integrate, Prove — the delivery discipline that turns a designed workflow into a system that survives real data, real users, and real failure.
This is the second half of a pair. The A.G.E.N.T. playbook — introduced by Hofmann and Kruhse-Lehtonen in the Harvard Data Science Review — covers the design half: auditing the workflow, gauging where agents belong, and engineering the redesign. S.H.I.P. is our own implementation playbook for everything after that decision.
Design is not delivery
A.G.E.N.T. answers what the workflow should become. It does not answer whether the thing you build will hold up when a field agent submits a half-finished record at 2am. Most agentic projects die in that gap — not because the design was wrong, but because the team moved from a whiteboard straight to an integration, and every subsequent failure had six possible causes. S.H.I.P. is the delivery discipline we run after the design is settled. It is ours, drawn from shipping production systems at Oracle, IBM, Protocol Labs, and Scotiabank, and from our own engagements since.
S — Scope the slice
Pick one path through one workflow and declare three things in writing before any tooling decision: where the agent acts alone, what conditions hand control back to a person, and the single number that will tell you this worked. The autonomy boundary is the hard part and the part teams skip. On KALAP we drew it deliberately — the field interview stays fully human and unscripted, because a person being interviewed behaves differently in front of a form, and structure is recovered by AI afterward. That was a scoping decision, not a technical one, and it determined everything downstream. Everything outside the slice is iteration two.
H — Harden the logic
Build the agent against inputs you wrote by hand, not against your data source. A hand-written input is a known quantity: when the output is wrong, the logic is wrong, and there is nowhere else for the fault to hide. Connect live data before this holds and every bug has two parents. Add one step, verify it, then add the next — chaining untested steps together buys an afternoon and costs a week, because a failure three steps deep gives you no signal about which step produced it. If the design calls for several agents, each one earns its place in the chain by working alone first.
I — Integrate under real conditions
Now swap the hand-written inputs for the live source, and expect the system to get worse. Real records arrive incomplete, inconsistently named, and in shapes nobody described to you. This is the phase where enterprise delivery habits pay for themselves: verification at each boundary, explicit handling for the partial record, and a rollback path where the domain allows one. Test with inputs that are unusual and malformed, not just the clean example that convinced everyone in the demo. A workflow that survives one ideal input has proven nothing. Then run it end to end and confirm the output actually reaches the person who needs it, in the form they can use.
P — Prove or stop
Measure against the baseline you committed to in Scope, not against a number chosen afterward to make the result look good. Three outcomes are legitimate: scale it, iterate on it, or stop. Stopping is a real result and costs far less here than after a year of quiet maintenance. What you learned — where the boundary was drawn wrong, which handoff created friction, which step should never have been automated — is the input to the next Audit. Run the loop again on the next workflow. The compounding advantage is the delivery muscle, not any individual system.
The build sequence
The order matters more than the tooling. Each step exists so that when something breaks — and it will — there is exactly one place the fault can be.
01
Write down what v1 must do
One sentence on the outcome, one on the autonomy boundary, one on the metric. If you cannot write these, you are not ready to build — you are still designing.
02
Prove the trigger in isolation
Confirm the entry point fires and carries the data you expect. Do not build on top of a trigger you have not watched work.
03
Build each agent step against a fixed input
Hand-written input, one step at a time. Verify the output format and content are genuinely what the next step needs, not merely plausible.
04
Prepare the data before you connect it
Check completeness, naming consistency, and that the fields the agent expects actually exist. Most integration bugs are data-shape bugs wearing a costume.
05
Connect live data and re-verify
Swap the fixed input for the real source and confirm the output holds. Where it degrades, you have found a real requirement.
06
Run it end to end, then check delivery
Trigger the whole path and confirm each step hands off correctly — then confirm the result lands where the user will look for it, in a usable form. An interface, if v1 needs one, comes last.
07
Write the improvement list
What broke, what you would change, what comes next. A prototype that surfaces three concrete improvements has done its job. That list opens the next cycle.
Where first builds actually fail
Not in the model. In the discipline around it.
Changing scope mid-build
Ideas surface once the thing is real. Capture them and keep going. Small corrections are fine; new features and new use cases belong to the next iteration, not this one.
Assuming models are interchangeable
The same prompt produces materially different output across models, and behaviour that held in a chat window can shift once it runs inside a workflow. Pin the model per step and re-verify whenever it changes.
Changing things at random when stuck
Every build hits a wall. Read the error properly, reduce the failing step to its smallest form, and test with a simpler input. Remove complexity before adding any.
Automating the wrong step
The most automatable step is often not the expensive one. If throughput does not move, check whether you accelerated something that was never the constraint.
Have a design that needs to ship?
We run engagements end to end — A.G.E.N.T. for the design, S.H.I.P. for the delivery.
