The Forced Choice Fallacy: Why Your Customer Surveys Are Probably Worthless

In partnership with

How many times have you heard this in a meeting?

"Our customer satisfaction rating is 4.7 out of 5! Our NPS score is 82! Customers LOVE us!"

...only to watch your actual sales numbers tell a completely different story.

Here's the uncomfortable truth: most of the survey data your company collects is practically useless for predicting what customers will actually do.

I stumbled across some fascinating research that explains why - and more importantly, how to fix it.

The Rating Scale Trap

Here's a question for you: On a scale from 1-10, how important is it that your next phone has good battery life?

If you're like most people, you probably answered 9 or 10.

Now, how important is it that your next phone has a great camera? Again, probably 9 or 10.

And a beautiful design? Fast performance? Good value?

See the problem? When there's no cost to saying everything is important, people will say... everything is important.

This explains why media companies found that people who rated themselves as "extremely likely" to see a movie often never showed up at theaters. There was no cost to their hypothetical enthusiasm.

The Psychology of Real Decisions

In the real world, every choice comes with an opportunity cost:

• Seeing Movie A means not seeing Movie B

• Buying iPhone means not buying Galaxy

• Choosing Hospital X means not choosing Hospital Y

Real decisions involve trade-offs, but standard surveys completely miss this fundamental aspect of human decision-making.

The Forced Choice Revolution

The solution is brilliantly simple: stop asking people to rate things and start forcing them to make choices.

Instead of: "How likely are you to see Mission Impossible: The Final Reckoning?" (1-5 scale)

Ask: "Is seeing Mission Impossible on your list of the top three things you plan to do this weekend?"

The results? When one media company switched to this approach, their predictive accuracy skyrocketed. While 15% of people might say they're "extremely likely" to see a movie, only 1-3% would put it in their weekend top three - and that smaller number actually predicted box office performance.

The Real-World ROI

This isn't just academic theory. Companies implementing forced choice methodologies are seeing dramatic improvements:

• A hospital network increased patient conversion by 34% after redesigning their marketing based on forced choice insights

• A tech company cut ad spend by 40% while maintaining the same conversion rate

• A streaming service accurately predicted their hit shows by asking "If you could only watch one new show this month, which would it be?"

Beyond Simple Questions

For more sophisticated research, techniques like MaxDiff and conjoint analysis take the forced choice concept to the next level:

• MaxDiff forces respondents to pick the most and least important features from sets of options

• Conjoint analysis presents complete product configurations at different price points and forces choices

These methods mathematically extract what truly drives decisions, not just what people claim is important.

The Implementation Framework

Ready to stop collecting useless data? Here's how to transform your approach:

1. Audit Your Current Questions

   • Look for any rating scales or importance questions

   • Ask yourself: "Does this question force a trade-off?"

2. Create Tension in Your Questions

   • Put desired options in competition with each other

   • Make respondents commit to specific actions, not hypothetical intentions

3. Test Against Real Behavior

   • Compare your new survey results against actual customer actions

   • Continuously refine your questions based on predictive accuracy

The Bottom Line

In a world of infinite options but finite resources, the companies that understand actual customer priorities (not just stated preferences) will win.

The next time someone in your company suggests another "rate this on a scale of 1-10" survey, you might want to ask them:

"If you could only know one thing about our customers that would predict their behaviour with 90% accuracy, would you rather keep your current survey or try a forced choice approach?"

Now that's a question worth answering.

Until next time...