From pilot to production: a staged rollout
Outcome of this lesson
You can write a pilot plan that a cautious operations manager would approve.
The four-stage rollout
- Shadow mode (week 0–1). The agent listens or runs on recorded/sample calls; humans still take every call. You compare what the agent would have said with what the human did.
- Limited pilot (weeks 1–3). Route a small, low-risk slice: for example after-hours calls only, or 10–20% of one call type. Review every call in week one.
- Expanded pilot (weeks 3–6). Increase traffic and add call types that passed your Automation Matrix. Review a sample daily.
- Production. Full routing for approved call types, human fallback always on, weekly review rhythm continues.
Go / no-go gates
Set numeric gates before the pilot starts. Example for a booking agent:
- Task success (booking completed or correct handoff) ≥ 85%.
- Made-up information: 0 incidents in reviewed calls.
- Transfer requests honoured: 100%.
- Complaint rate ≤ the human baseline.
- Median response gap under about one second.
Miss a gate? Stay at the same stage, fix, and re-test. Do not expand on hope.
Rollback plan
- One switch that sends all calls to the human line or voicemail.
- A named person who may press it, 24/7 during the pilot.
- Triggers: repeated hallucination, outage, complaint spike, compliance concern.
Tell people
Staff should know what is changing and why; customers hear an honest introduction on every call. Hidden pilots cause mistrust.
Do this now
Write your four stages with traffic percentages, dates, gates and a rollback switch.
💡 Real-world example
Illustrative scenario. An outsourced support team piloted on “order status” calls only — the safest, highest-volume type. After four weeks with gates met they added “returns eligibility”. They never sent a billing dispute to the agent. Scope grew only when evidence said it could.
✅ Checklist
🛠 Your assignment
Decide who owns the rollback button and write their phone number on the plan.
📝 Check your understanding
Answer 2 of 3 correctly to complete this lesson.