Short answer
A confidence threshold is the score an AI-drafted support reply must reach before it is sent automatically. Replies above the threshold go out in seconds; replies below it wait in a review queue for a person. Combined with hard rules (refunds and complaints always reviewed), it lets D2C brands automate routine replies without risking wrong answers on sensitive cases.
Key takeaways
- A threshold turns 'should AI reply?' into a measurable, adjustable rule.
- Start high, review drafts, then lower it intent by intent.
- Hard rules sit above the threshold: some intents always need a person.
- Watch reopen rate and CSAT on automated replies to know if the threshold is right.
The biggest fear about AI in customer support is simple: what if it says something wrong? Confidence thresholds are the main way to manage that risk without giving up automation.
What a confidence threshold is
When an AI agent drafts a reply, it also produces a confidence score based on how clearly it understood the intent, whether it found the data it needed, and whether the answer follows your policies. The threshold is the line:
| Score | What happens |
|---|---|
| Above threshold | Sent automatically, in seconds |
| Below threshold | Queued for a person to approve, edit or reject |
Hard rules sit on top
Some cases should never auto-send, however confident the AI is:
- refund approvals and payment disputes;
- damaged, wrong or missing items;
- legal, safety or health concerns;
- very negative sentiment or threats to post publicly;
- VIP or wholesale customers, if you treat them differently.
How to set your first threshold
- 1
Start in draft-only mode
For 1–2 weeks the AI drafts everything and nothing sends automatically.
- 2
Measure edit rate by score band
For drafts scoring 90+, 80–90, 70–80, how often did your team send them unchanged?
- 3
Pick the band where edits are rare
That becomes your starting threshold for safe intents.
- 4
Turn on auto-send for one intent
Usually order status first.
- 5
Review weekly
Check reopen rate and CSAT on automated replies; adjust.
Signs your threshold is wrong
| Signal | Likely meaning |
|---|---|
| Reopen rate on automated replies rising | Threshold too low; replies send but don't resolve |
| Review queue overflowing with correct drafts | Threshold too high; you're wasting team time |
| CSAT lower on automated replies | Tone or completeness problem; check instructions |
Thresholds per intent
Different intents deserve different thresholds. Order status with clean tracking data can run at a lower threshold than a product question about allergies. Set them separately.
How MarQet CX handles it
MarQet CX scores every draft and sends only what clears the threshold you set; refunds, angry customers and anything unclear wait for your team in the review queue.
Frequently asked questions
What is a confidence threshold in AI customer support?
The minimum confidence score an AI-generated reply needs to be sent automatically. Below it, the reply waits for human approval.
What threshold should I start with?
Start high and use 1–2 weeks of draft-only data to find the score above which your team rarely edits replies, then set the threshold per intent from there.
Which support replies should always need a human?
Refund approvals, damaged or wrong product complaints, legal or safety issues and very negative sentiment.
Written by
Ayush Singhal, Founder, MarQetAyush founded MarQet to give D2C founders one system for customer conversations, daily performance and quick-commerce shelf availability.