AI customer support: an honest guide to what works

AI customer support is sold with a straight face as "resolve most of your tickets automatically" and dismissed with equal confidence as "chatbots that infuriate everyone". Both takes are lazy. Modern AI agents are genuinely good at a specific slice of support work, genuinely bad at another slice, and the teams getting value are simply the ones who drew that line honestly. This guide is the line-drawing exercise, from a company that ships an AI agent and would still rather set expectations than inflate them.

What today's AI agents actually do well

• Questions with documented answers. "How do refunds work", "where do I change my password", "what does the Pro plan include". If a good help article exists, a retrieval-based agent finds it and answers accurately in the customer's language, at any hour, instantly.
• The volume tier that burns teams out. The same twenty questions arriving hundreds of times is exactly the work humans do worst (boredom breeds errors) and AI does best (consistency is free).
• Off-hours coverage. An honest "here is the documented answer, and a human will confirm the edge case in the morning" beats an empty inbox. (Where the line sits between an AI agent and an old-style scripted bot is its own question: live chat vs chatbots.)
• Triage and drafting. Even where the AI should not answer, it can categorize, summarize long threads, and draft replies for human review. Copilot-style usage is underrated and low-risk.

What they are still bad at, and will be for a while

• Anything requiring account judgment. Refund exceptions, goodwill credits, "this customer is about to churn" reads. These need context and authority an agent should not have.
• Undocumented territory. An AI with no source either declines (fine) or improvises (catastrophic). The failure mode of a confident wrong answer about billing is worse than no answer.
• Angry, complex, or multi-thread situations. A customer three messages deep in frustration needs a human name in the reply, immediately.
• Being your excuse. Deflection dressed up as automation, where reaching a human takes detective work, converts support cost into churn cost at a terrible exchange rate.

The setup that decides everything: your help content

The single biggest predictor of AI answer quality is not the model; it is the documentation the agent retrieves from. Symptom-phrased titles, answer-first paragraphs, one task per article: the same rules that make articles findable by humans make them retrievable by AI. If your docs are thin, fix that before turning anything on; the difference is dramatic. Our guide on writing help articles people actually find doubles as the AI-preparation checklist.

The rules of honest deployment

1. Label it. Customers forgive an AI for being an AI; they do not forgive being tricked. A clear "AI assistant" label with visible escalation costs nothing and buys trust.
2. Make the human path one message wide. "Talk to a human" should always work, and the human should receive the full transcript so the customer never repeats themselves.
3. Constrain it to sources. Retrieval from your published content, with the confidence to say "I do not know, routing you to the team". Tune the handoff threshold conservative at first.
4. Review transcripts weekly, early on. The wrong answers cluster fast, and each cluster is either a docs gap (write the article) or a scope error (tighten the boundary).
5. Measure the pair, not the single number. AI resolution rate only means something next to the CSAT of conversations that were handed off. High "deflection" with unhappy handoffs is failure wearing a success metric. (More on measurement in customer service metrics that matter.)

The economics, stated plainly

Pricing models shape behavior. Per-resolution pricing (Intercom's Fin charges $0.99 per resolved query) means your costs rise exactly when volume spikes, and creates a quiet incentive for the vendor to count generously. Included-with-credits models keep the bill flat and the incentives clean. We are biased here and say so: Chattypie's AI agent is included in paid plans with usage credits. Whatever tool you choose, model a busy month before signing.

A realistic rollout plan

1. Week 1: Fix the top ten help articles. Turn the AI on for after-hours only, watch every transcript.
2. Week 2 and 3: Expand to business hours for documented categories. Keep billing exceptions and cancellations human-only. Write articles for every question the AI declined.
3. Week 4: Compare the pair: automated resolution rate and post-handoff CSAT. Loosen or tighten scope based on evidence, not vibes.

Run it this way and the honest outcome for most teams is meaningful: the repetitive tier answered instantly around the clock, humans concentrated on conversations that need judgment, and a documentation habit that compounds. That is less romantic than "AI resolves everything" and considerably more true.