How Chatbot Metrics Influence Customer Service Outcomes

Many businesses are hesitant about adding a chatbot to their websites. Will it improve customer service metrics enough to justify the expense? Do chatbots even improve customer service at all? To find this out, you need to look at chatbot success metrics that specifically tie chatbot performance to your customer service outcomes.

How to Evaluate Chatbot Performance

There are three main indicators of customer service performance. These are:

1. Customer satisfaction
2. Customer service team capacity
3. Customer service agent satisfaction

Let’s break down the most important chatbot metrics for each of these qualitative outcomes.

1. Customer Satisfaction

A study published in the Journal of Targeting, Measurement and Analysis for Marketing found that there is a direct link between retention and word-of-mouth brand referral. In particular, CSAT performance is a significant revenue factor.

Chatbots contribute to measuring customer satisfaction on a micro and macro level in a few ways.

The Chance to Review Individual Messages

Modern chatbots are able to handle around 80% of routine tasks as well as 30% of all live chat, according to Master of Code. You can find out what this looks like for you by checking your unanswered messages in your dashboard.

However, if you want to understand the customer experience, it’s necessary to turn to individual messages and their responses. You should specifically examine:

• What the customer wanted
• How the chatbot responded
• How the response from the chatbot fitted into the overall conversation

Combining this with message feedback and the end-of-conversation net promoter score (NPS) gives you valuable insights into your customer service chatbot performance. It also helps you identify the specific pain points your customers are experiencing. You can then take action to improve performance based on this data.

Message Feedback

Many chatbots allow customers to leave feedback on specific messages using a simple thumbs up or down. This helps you pinpoint underperforming chatbot replies. Use the conversation view to provide context. You’ll then be able to improve content in chatbot replies for specific intents, topics, and questions. You may be able to improve chatbot feedback simply by improving the copy, by adding an image, GIF, or video to provide more detail, or by linking to a page on your website that has more information.

NPS and Conversational Feedback

A Net Promoter Score is a short, two-part survey that businesses often offer at the end of a customer service conversation. The first part quantifies customer satisfaction by asking customers if they are likely to recommend the product or service to their friends or colleagues. The second part is a qualitative, free-form text where customers can leave a comment about their experience.

By offering NPS surveys in chatbot conversations, you’ll receive a better idea of the average customer experience on a referral level. The advantage of using NPS at multiple touch points is it identifies which individual services need extra attention to improve overall performance.

2. Customer Service Team Capacity

As an example of why customer service team capacity is significant in the customer experience, consider Amazon. A capacity planning solution found how to save more than a million dollars for the customer service department. Part of this came down to creating a model for diversifying the messages the customer service team was answering.

“These advancements in planning our capacity and optimizing our contact allocation plans have significantly improved our ability both to respond to customers quickly, which improves customer experience, and also to lower our costs, which increases corporate flexibility.” — Kim Rachmeler, Former VP of Worldwide Customer Service at Amazon.

Some key chatbot KPIs will allow you to measure whether you are seeing increased customer service team capacity from chatbot implementation.

More Conversions Addressed by AI

One is how many more customers you serve through your conversation support. Every brand that implements chatbots should expect an increase in conversation volume capability. This is because an AI engine that talks to your customers with no regard to queues is able to manage more conversations.

Brands that extend their service hours to 24/7 conversational support typically see their conversation volumes increase by about 40%. The metrics from all-day conversational support will give you insights into the ideal time-of-day availability for your customers.

When chatbots are only active during live chat service hours, there is still an increase in conversation capacity. However, it is only about 10% to 15% (based on companies with around 300,000 annual customers).

Queue Length and First Response Time

As Kim Rachmeler points out, reducing the amount of time a customer has to wait to speak to a representative has a positive impact on the customer experience. However, do customers spend more or less time waiting for service when you have a chatbot?

Chatbots are extremely efficient at answering frequent customer issues. Plus, they do so concurrently — answering as many as 100 customer questions in the same time a customer service advisor could only answer one.

However, this only applies to frequent questions and the automated processes you want AI chatbots to handle. The remaining questions will still go to your human customer service agents.

Depending on your business, a chatbot may be able to resolve anywhere from 60% to 90% of all your chat issues, meaning 10% to 40% will be transferred to a human agent. This could be through live chat or through the chatbot opening a ticket with your customer service ticketing system.

Although a typical chatbot performance metric to use is total chat conversations, it makes sense to look at your entire customer service offering:

• Are you seeing fewer frequent issues in your phone and email support?
• Has there been a reduction in calls, email, or form submissions as chat numbers have increased?
• Do your agents now have more time to engage in activities that will improve your overall customer service?

Varma saved 330 hours a month, which was enough to give two of its service agents new responsibilities.

Tina Kurki, Senior Vice-President of Pension Services and IT, said this about the transition: “Our GetJenny chatbot, Helmi, complements our customer service department, the quality of our telephone customer service has changed; common issues are reduced, while calls requiring human expertise are dominating.” (GetJenny is a chatbot solution developed by LeadDesk.)

3. Customer Service Agent Satisfaction

There are many factors contributing to agent satisfaction — from team dynamics to management quality. However, the one that likely contributes most to agent stress and turnover is repetitive and continuous human interaction coupled with strict quantitative targets, according to Keser and Yilmaz.

It’s important to keep in mind that the quality of interactions is tied to a sense of accomplishment. “Agents report that the most satisfying part of the job is when the call ends in the customer’s needs having been met. If the issue was a particularly challenging one, the emotional satisfaction of a job well done and a customer satisfied are factors that keep call center employees coming back for more of what can be a very stressful role.” — Susan Leah in Increasing Customer Satisfaction Through Employee Satisfaction in a Call Center Environment.

This is, again, an area where chatbots have an impact. What chatbot performance metrics are relevant to elevating agent satisfaction?

Deflection and Automation

When a chatbot works at the same time as agents (during either live chat or phone calls), it reduces the number of conversations agents need to handle. This is called deflection. A chatbot working outside of service hours (and answering customer questions successfully) is called automation.

We’ve already considered chatbot deflection and automation rate — it’s that 60% to 90% statistic from above. The exact percentage you’ll experience will depend on:

• The quality of your chatbot
• The types of question you allow your chatbot to answer
• The industry, use case, and complexity of your product

Automation and deflection metrics are important because they are an indication of the impact of chatbots on customer service agent satisfaction. Deflection and automation reduce the number of repetitive, simple questions agents encounter in their everyday work. This gives them extra time to concentrate on more complex customer service issues.

Robotic Process Automation (RPA) Adoption

Reducing frequent issues is often the first step in improving customer service operations. The next step may be to introduce RPA into your AI chatbot implementation. This means AI takes over procedures that are more complicated than answering a simple question but are still tedious enough to be a chore for customer service agents. Some examples of such tasks include:

• Providing tracking and delivery information
• Letting a customer know if a certain product is in stock
• Cancelling an order for a customer

These tasks don’t just require knowledge; they also require the capability to perform actions. The action may be:

• Verifying the identity of the customer
• Checking internal systems
• Finding out information and relaying it to a customer
• Taking another type of action based on a customer’s need

When you use RPA to automate these actions — like A-Lehdet did — you remove the burden from the human agents who have to perform these actions repeatedly.

Creating a Working Environment Where You Value Knowledge and Critical Thinking

Deflection, automation, and robotic process automation all directly relate to Susan Leah’s point: that more challenging customer issues give agents a greater sense of accomplishment. In other words, handing off monotonous tasks to a chatbot transforms the position of customer service agent from a repetitive job to a career where subject matter expertise, critical thinking, and empathy are valued.

Summary (TL;DR)

AI chatbots will transform your customer service — internally and externally — by significantly improving your customer and employee experience.

Chatbots allow you to improve customer satisfaction and measure it more frequently. Bring chatbot metrics into your overall CSAT score improvement plans with these KPIs:

• Unanswered messages
• Message feedback
• NPS and conversation feedback

Customer service team capacity planning is a challenging aspect of operations. Chatbots will boost your capacity while providing you with data to plan your future strategy. Use these metrics and compare them with your other channels:

• Conversation count
• Queue length and first response time

Agent satisfaction is essential for forming a committed customer service team. Any brand that cares about customer experience should be promoting knowledge and skill development in its teams, reducing agent churn, and creating an environment that builds confidence and celebrates achievement. After all, your frontline support is the face of your brand. Focus on improving these metrics to give your agents a more rewarding workplace:

• Deflection and automation rate
• Actions automated by RPA

LeadDesk helps brands connect with their customers 24/7, using AI chatbots tailored to improve the customer experience. The user-friendly approach and simple-to-use interface allows you to focus on serving customers at a new scale.

Customer service agents love the LeadDesk chatbot because it deflects the repetitive, tedious tasks and gives them the chance to do what they do best: provide an amazing service.

Leverage AI chatbots to improve your customer satisfaction, customer service capacity, and agent satisfaction. Book your 20-minute demo.