How to Roll Out an AI Support Agent Safely (Step by Step)

Short answer: roll out an AI support agent the way you would onboard a new hire. Give it one topic, only approved answers, a written list of things it must never promise, and a period where it drafts and a person sends. Widen its scope only when the drafts are right, and know how to take a topic back before you need to.

The fear most support leads have is not that the AI is slow. It is that it says something nobody on the team knows about: a refund date, a callback, an SLA. This plan is built around preventing that.

The rollout checklist#

Step	What to do	How it works in SupportWire
1. One topic	Pick one repetitive topic the agent owns. Route everything else to people.	Pick the topics Kal owns and the ones it routes.
2. Approved sources	The agent answers only from content your team approved.	Kal reads from the Knowledge Store. New answers arrive as drafts and a person approves them first.
3. Promise rules	Write down what it must never promise.	Rules such as Never promise SLAs, Never quote prices, Never mention competitors, and a rule for refund timelines.
4. Draft mode	Run it on real conversations where a person sends every reply.	Put a channel in draft mode so Kal writes for your team instead of sending.
5. Confidence threshold	Below a set confidence, it hands off instead of guessing.	Drag the confidence dial. Handoffs carry the full thread.
6. Widen slowly	Add the next topic or customer segment only when drafts are right.	Scope is set per topic and per segment.
7. Rollback	Know how to switch it off per topic before launch day.	Turn Kal off per topic, per segment, or all at once.
8. Review	Read what it said and why, every week.	Every reply is logged with citations, confidence and edit history.

Step 1: start with one topic#

Pick one topic that is frequent, has a clear right answer and low risk if it goes wrong. "How do I reset my password?" is a good first topic. "Can I get a refund?" is not, because the answer depends on the customer.

Everything outside that topic goes to people as it does today. You are testing the agent, not replacing the queue.

Step 2: approved sources only#

An agent that answers from the open web or from a model's memory will eventually make something up. Limit it to sources your team wrote and approved.

In SupportWire, Kal reads from the Knowledge Store, and every reply cites the source it used. New knowledge (from Slack threads or resolved conversations) arrives as a draft, and nothing reaches a customer until a person approves, merges or rejects it. If an answer is wrong, you fix the source once instead of correcting the agent conversation by conversation.

Step 3: write the promise rules before launch#

Most bad AI support moments are promises: "you'll get a callback tomorrow", "your refund is on its way", "we'll fix this by Friday". Decide up front which of these the agent may say.

A useful starting list:

• Never promise an SLA or response time.
• Never quote a price that isn't in an approved source.
• Never promise a refund timeline unless your policy states it.
• Never promise a callback or a follow-up a person hasn't agreed to.

Kal's Rules screen has these as guardrails (Never promise SLAs, Never quote prices, Never mention competitors) plus a rule for whether it may promise refund timelines. When work actually needs to happen, such as a bug, Kal opens the Linear issue itself with repro steps rather than telling the customer "I'll file that for you".

Step 4: run it in draft mode on real conversations#

Testing on made-up questions tells you little. Real customers phrase things in ways nobody predicts.

Put the first channel in draft mode. Kal writes a reply for each conversation in scope, and a teammate reads it, edits it if needed and sends it. Each edit is a signal about what to fix: a missing source, a rule that's too loose, a topic that isn't ready.

Stay in draft mode until your team is mostly sending drafts unchanged. That is the bar for letting it send on its own.

Step 5: set a confidence threshold and a clean handoff#

Even inside one topic, some questions are unusual. The agent should hand those off instead of guessing.

In SupportWire you set a confidence dial. Below it, Kal hands the conversation to a person with the full thread and what was already tried, so the customer doesn't have to repeat themselves. See smart handoff for how that context travels.

Step 6: widen one topic or segment at a time#

Once the first topic runs well, add the next one. You can also widen by customer segment, for example letting the agent answer self-serve customers first while larger accounts keep a person on every reply.

Each new topic goes through steps 2 to 5 again: sources, rules, draft mode, threshold.

Step 7: know how to roll back#

Before launch day, make sure anyone on the team knows how to take a topic away from the agent. If a policy changes or something goes wrong during an incident, you want to pull back one topic, not switch off support.

Kal can be turned off per topic, per segment or all at once.

Step 8: review every week#

Read a sample of what the agent said, not just a dashboard number. In SupportWire every reply is logged with its citations, confidence and edit history, and your team can override any answer. When you change something, Kal replays the change against past conversations before it goes live. Gap Radar groups the questions Kal couldn't answer into knowledge gaps, so you can write the missing source once.

Common mistakes#

• Launching on every topic at once. You can't tell which topic is causing bad answers.
• Skipping draft mode because the demo looked good. Demos use clean questions.
• No written promise rules. The agent fills silence with reassurance, and reassurance becomes a commitment.
• No rollback plan. The first incident is the wrong time to find the off switch.

Where to go next#

• How Kal is scoped and controlled
• Smart handoff to humans
• How to teach an AI support agent from your team's answers