What is sandbox testing vs. live testing?
Two environments for evaluating an agent: controlled sandbox and real production traffic.
Updated June 2026
Sandbox testing and live testing are two environments for evaluating an AI agent. Sandbox testing runs real customer questions against the agent in a controlled setting; live testing evaluates it on real conversations in production.
Both should use genuine customer queries, including complex, vague, edge-case, sensitive, and multilingual scenarios, to get an accurate read on business performance and conversation quality.
See it on SupportWire
Related terms
- Continuous improvement loop A structured cycle: train, test, deploy, analyze, repeat. Systems that learn by default.
- Lightweight governance A low-friction way to keep iterating on an AI agent without bureaucracy.
- Proof of concept A focused evaluation of one strongest-fit agent in a realistic setting, not a multi-vendor bake-off.
- Human involvement rate How often humans step in, and whether those moments are genuinely high value.