Takeaway: An agent that works is not the same as an agent that is safe to speak or act on behalf of the company.
A working agent answers your test prompts. A safe agent survives every customer, with trust.
This is the largest canyon in the journey from internal AI to customer-facing AI. The gap between those two is where most customer-facing AI projects quietly stall — or loudly fail. Internal usefulness is a low bar. Customer exposure is a different category of risk: legal, reputational, financial, and emotional.
This article is about the second real challenge in AI-native adoption: the trust factor between what the team has built and what the customer should be allowed to touch.
The Signal
The pressure to ship customer-facing AI is real. The customer appetite for it is not. And the companies that ran hardest at it are already pulling back.
Gartner’s 2026 survey of customer service leaders found that 91% are under pressure to implement AI this year. (Gartner, 2026)
Then there is the gap between internal AI and customer-facing AI. McKinsey’s State of AI 2025 found 88% of organizations now use AI somewhere, but no more than 10% in any given function say they are actually scaling AI agents — and customer service suffers the worst from integration gaps, producing “fast conversations, not faster outcomes.” Internal copilots are everywhere; autonomous customer-facing agents are rare. (McKinsey, 2025)
And the public pullbacks have started. In early 2024, Klarna claimed its AI assistant was doing the work of 700 customer service agents and handling 75% of customer chats. By 2025, CEO Sebastian Siemiatkowski publicly admitted the company had “focused too much on efficiency and cost,” that quality dropped, and that Klarna was rehiring humans into a hybrid model. (Entrepreneur, 2025; CX Today, 2025)
Executives are pushing to deploy. Customers are pushing back. Even the most public bets are walking it back. The narrow path between those pressures runs straight through trust.
The Support Engineer Robot
I have seen this debate up close.
Our support engineering team recently built an AI assistant for triaging and answering customer issues. It was actually pretty good.
It read incoming tickets, pulled context from our docs, summarized the customer’s product setup, looked up prior issues, and drafted clean responses. Internally, it cut a junior engineer’s research time on a ticket from twenty minutes to two.
The team was excited. The next question came up naturally: Can it just respond to the customer directly?
That is where the room got quiet.
Because internal usefulness is not the same as customer-facing trust. The agent worked in the conversations we tested. We did not yet know:
- What happens when a customer pastes a prompt-injection attempt into a ticket?
- What is the confidence threshold for the agent to even try answering? When should it hand-off to a human?
- What permissions does the agent need to actually resolve anything, and what is the blast radius if those are abused?
- Who is accountable if the agent makes a promise the company cannot keep?
And the biggest question of all: Do customers even tolerate interacting with the agent?
We did not have clean answers. So we did not put it in front of customers. Not yet.
That was not weakness. It was the right call. The agent moved into a human-reviewed draft mode — the engineer kept the last click (for now). That single guardrail let us keep the leverage while we worked on closing the trust gap.
Output Trust vs. Operational Trust
The trust gap is actually two gaps stacked on top of each other.
Output trust is whether the agent says the right thing in a typical conversation. Most teams test this. It is the demo bar.
Operational trust is whether the agent behaves safely across every conversation: hostile inputs, edge cases, ambiguous policy, missing context, broken upstream systems, and the weird 1% that real customers create every day.
A demo passes ten test conversations. Production sees ten thousand. The difference is not skill — it is distribution.
Output trust is necessary. Operational trust is what makes the agent safe to expose.
What “Safe Enough” Actually Requires
A customer-exposed agent is not a model with a prompt. It is a system with boundaries.
The boundaries that matter:
- Bounded data access. The agent only sees what it needs for the task. Not the whole CRM. Not all tickets. Not other customers’ data.
- Bounded tool access. Each tool call has a defined scope, rate limit, and reversibility check. Write actions need explicit approval paths.
- Adversarial input handling. Prompt injection, jailbreaks, and policy bypasses are tested as part of evals, not assumed away. OWASP’s Top 10 for LLM Applications lists these as first-class risks for a reason. (OWASP)
- Source-grounded answers. Claims are tied to retrieved, current, citable sources. The agent declines or escalates when it cannot ground an answer.
- Regression evals. A held-out set of customer scenarios must keep passing on every prompt, model, or tool change.
- Audit trails. Every conversation captures inputs, retrieved context, tool calls, model version, and final output. Reproducibility is not optional.
- Escalation paths. Clear rules for when the agent stops and hands off to a human, with full context attached.
- Rollback. A way to disable the agent or roll back to a prior version in minutes, not days.
- Tool and MCP controls. If the agent uses MCP servers or external tools, the OWASP MCP Top 10 risks — tool poisoning, indirect prompt injection, excessive permissions — need explicit treatment. (OWASP MCP)
This is a security and reliability checklist, not a model checklist. The NIST AI Risk Management Framework treats this as the gap between trustworthy and deployed, and expects organizations to manage it across the lifecycle, not only at launch. (NIST)
Pro Tip; AI Agents are actually very good at generating their own corner cases.
— Build a test agent that takes your chatbot through adversarial scenarios, emotionally charged conversations, policy edge cases, …
What Customers Will Actually Tolerate
Even a perfectly engineered agent runs into customer acceptance.
Customers will tolerate AI for:
- order status, shipping updates, account lookups
- simple FAQs and policy questions
- routing and triage
- self-service tasks they could already do alone, but faster
Customers expect a human for:
- emotional or sensitive situations
- complex or unresolved issues
- contractual, legal, financial, or compliance topics
- anything where they are already frustrated
Pro Tip: It’s always accepted for an agent to say: “While I’m connecting you to a human, here is what I found so far…”
— that is a trust-building handoff, not a failure.
This is not irrational. It is calibrated. People know what a bot is good for and what it is not. Hiding the bot, denying escalation, or burying the “talk to a human” button breaks trust faster than a wrong answer ever will.
The escalation path is not a fallback. It is a feature.
When AI Goes Public, Badly
The cautionary tales are public and recent.
Air Canada (2024). A customer-facing chatbot gave a passenger incorrect bereavement-fare information. The Canadian tribunal ruled the airline was responsible for its chatbot’s statements and ordered it to honor them. The company owns what the agent says, full stop.
DPD (2024). The delivery company’s customer service chatbot was manipulated by a user into swearing at them and writing a poem about how bad the company was. The screenshots went viral within hours. Adversarial inputs are not a corner case; they are the first thing a curious customer will try.
Chevrolet of Watsonville (2023). A dealership chatbot was prompt-engineered into “agreeing” to sell a car for one dollar, with a “no takesies backsies” clause. The clip spread across the internet. An agent without bounded tool access and policy guardrails will follow the conversation wherever it leads.
None of these failed at the model layer. They failed at the system layer: missing guardrails, missing escalation, missing accountability, missing tests for hostile input.
That is the trust factor.
The Customer Exposure Gate
Before putting an agent in front of customers, decide which level of exposure you are actually ready for.
Level 1 — Internal Assistant.
Employees use it. No customer ever sees its output. The bar is usefulness and basic security. Failure is annoying, not damaging.
Level 2 — Human-Reviewed Draft.
The agent drafts; a human approves before the customer sees anything. Captures most of the leverage. Limits blast radius to whatever the human catches. The right default for most teams starting out.
Level 3 — Limited Customer Self-Service.
The agent answers bounded queries directly — status, FAQ, simple tasks — and escalates everything outside its scope. Requires real evals, adversarial testing, audit logs, escalation paths, and rollback. Scope is the safety belt.
Level 4 — Autonomous Customer Response and Action.
The agent handles full conversations and takes actions on behalf of the company. Requires everything above plus strong identity and permission controls, reversibility on actions, continuous monitoring, defined accountability, and a tested incident plan.
You do not get to Level 4 by being optimistic.
You get there by passing Level 3 boringly for a long time.
A Release Checklist for Each Level Up
Before promoting an agent from one level to the next, get clear answers:
- What is the smallest scope the agent needs to be useful at this level?
- What data, tools, and permissions does the agent have, and why each one?
- What is the eval set, and what is the pass bar?
- How are adversarial inputs tested, and what coverage do you have?
- What grounds every customer-facing claim?
- What triggers escalation to a human?
- What is the audit trail, and who can read it?
- What is the rollback plan, and how fast can you execute it?
- Who is the accountable owner for this agent in production?
- What customer-visible disclosure tells users they are talking to AI and how to reach a human?
If you cannot answer these for the current level, you are not ready for the next one.
Pro Tip: Don’t write the adversarial test set by hand.
— Point a second AI agent at your chatbot and have it generate hostile prompts, prompt-injection attempts, emotionally charged conversations, and policy edge cases at scale. Score the results, fold the failures back into your eval set, and re-run on every prompt, model, or tool change.
The Point
The trust factor is not a model problem. It is a system problem.
It is also not a single decision. It is a gate with levels, and each level demands a different operating posture: bounded scope, evals against adversarial inputs, audit trails, escalation, accountability, and a customer experience that respects when humans matter more than speed.
Customers do not need your agent to be perfect.
They need it to be predictable, bounded, and honest about its limits — with a human reachable when it matters.
The agent works is the start of the conversation.
The agent is safe is what ships.
Related reading: