Why a Thumbs Up/Down CSAT Is a Bad Idea

We’ve been reviewing your customer satisfaction data, and we’ve got some bad news. If you’re measuring CSAT with a thumbs up/down methodology, your Voice of the Customer (VoC) program is encouraging customers to lie to you. 

Ok, they’re not purposely lying. But they are likely misleading you with artificially high numbers. This is because, despite what you might see on the news, people are generally nice… even to customer service. When given a binary choice on how they feel about an interaction, customers on the fence will opt for a more positive response. They feel good, you feel good, but then they leave you—and you’re left wondering why. What could you have done differently? This is what we call “smiling as they walk out the door”. 

Let’s examine why binary measures fail to deliver results, and how customer service can build better customer understanding using the popular, and customer-centric, VoC model: Ask, Analyze, and Act.

Step 1: Ask

Customer understanding begins with the questions we choose to ask, and for many service leaders, the standard has long been customer satisfaction, increasingly moving toward customer effort score (CES) and value enhancement score (VES). No matter the metrics you choose, the way we evaluate the voice of the customer should reflect how the customer actually felt about their experience. 

Unfortunately for service leaders, certain vendors not only allow but recommend that you measure CSAT as a binary output. Why? Because it allows them to present artificially high numbers, inflating the value of their platform. And because–let’s get nerdy for a second–it’s easy to put into a machine learning algorithm and run a logistic regression. But, logistic regression has a major limitation. This type of analysis assumes linearity between the dependent variable and the independent variables.

In human terms, that means satisfaction is not binary. In the real world of customer service interactions, the true answer to how satisfied a customer is with your service is most often “kinda.” Without the option to select “kinda,” in either direction, brands will never be able to truly understand how a customer feels about their service experience.  

So, how can you get a truer sense of your customers’ satisfaction? 

Start with the end in mind. Why are you asking customers for feedback? The whole point of asking customers to answer a survey question is to use their input to make improvements to your business. Therefore, you must ask questions that give you reliable data that the brand can act upon.  

From that starting point, the fix is actually quite simple. Change your survey to a 5 or 7 point Likert scale, which delivers a deeper level of insight by giving respondents choice without overwhelming them—while at the same time allowing for a lower margin of error.  

Step 2: Analyze 

To ensure that data can be used to drive operational decisions, we must understand what customers are trying to tell us. Again, without the ability to evaluate the spectrum of customer emotions, service leaders can only learn so much about the relationship between experiences and potential fixes. 

With a Likert scale, you’ll be able to assess the relative strength of the experience of any particular customer, or group of customers, against other variables. Combining algorithmic learning with human intuition will help you to evaluate how different aspects of the customer journey impact business outcomes like customer loyalty. You will also be able to identify nuances that can be used to build customer personas as you observe how customers from different demographics react to certain situations. 

Step 3: Act

With proper planning, this final step should be clear. By asking questions that give you insight into decisions you’re planning to make, actions should be clear. However, with binary satisfaction questions, you’ll be stuck only being able to make directional decisions. 

For example, you might see that the majority of customers prefer a certain channel or experience, but you can’t be sure how strong that feeling actually is. When using a scale, leaders can pinpoint which company actions have the most potential to impact customer satisfaction. 

If you’d like to better understand your customers and what drives their experiences, don’t sell your VoC program short by using misleading metrics that hinder your ability to analyze behaviors and act upon the findings. Say goodbye to the binary and move to a more nuanced way of evaluating your customer sentiment.


Devin Poole

Senior Director of Strategic Marketing at Dixa

Share this article

Get CX insights, proven strategies, & the latest news in your inbox every month!

You might also enjoy

AI in Customer Service
7 min read

5 metrics to track the impact of AI on your customer service

Read more
AI in Customer Service
7 min read

7 benefits of AI knowledge bases for increasing customer service efficiency

Read more
AI in Customer Service
7 min read

5 ways to boost customer retention with AI and measure it

Read more