Mitigating Biases in User Research

Cognitive biases are unconscious mental shortcuts that make your brain arrive at a conclusion without conducting a thorough analysis. It is ambient and can occur much more when we rely on intuitive judgement in tasks that are familiar, that we execute from a place of cognitive ease. Biases are not inherently harmful, they just cause irrational behavior thus concealing real intent or true self.

“The confidence people have in their beliefs is not a measure of the quality of evidence but the coherence of the story their mind has managed to construct.” #DanielKahneman

Biases pose a significant challenge in user research. A designer, user researcher or anyone interested in understanding the true behavior and motivations of a target audience trusts in user research to surface genuine insights. Unfortunately, everyone involved in the research equation is susceptible to bias. This can compromise data collection, cause inaccurate analysis and inferences.

The first step in making an attempt to limit biased user research and design outcomes is to recognise that it’s difficult for involves parties to act impartially. In addition, we can only proactively deal with known quantities thus it is important to understand some common biases that you will encounter during user research.

Here are some common ones and you can learn more from The Decision Lab site:

Status - we constantly look for ways to improve how others see us.
Social desirability - we constantly make ourselves look good to others.
Transparency illusion - we overestimate the extent to which other people understand what we want to convey during communication.
Analysis paralysis - our capacity to process new information and make decisions declines with each choice made.
Confirmation bias - we look for insights that support our existing beliefs and try hard to ignore the ones that challenge our perspectives.
Framing - we make decisions based on what is presented to us.
Authority - we tend to comply and placate those in authority.
Anchoring - what we see first affects our judgement of what comes next.
Implicit bias - we bear pre-emptive attitudes and stereotypes that shade our interactions, interpretations and how we associate with others.

You will notice that some affect the researcher, others the respondents and some impact both though in different ways. Some are prevalent in qualitative research like interviews and focus groups while others predominantly affect quantitative instruments like surveys. Now read them again and pause to reflect on how each impacts how you ask questions, who you ask questions to and how you would setup the context and environment for user research and data collection.

So what can we do about it? Setting the right mindset is key. Check whether your approach is truly testing the assumptions, feasibility and desirability of your product or you are merely seeking validation from potential customers.

A validation mindset wants to hear what they have done right and ignores any feedback that disapproves their assumptions. A testing mindset suppresses their presuppositions and looks at research as an opportunity to learn.

A validation mindset is ‘salesy’ and pushes users into futuristic hypotheticals that are not based on any observed behavior. A testing mindset listens first, observes how the user behaves when questions are asked and is more interested in a proven history of habits and customer behavior.

“Even your mother, who is biased to protect your feelings, would give you honest feedback if asked the right way.” #RobFitzpatrick

Lastly, you should rely on established behavioural design model to design your research instruments. This is the best way to limit your influence in research instruments by relying on frameworks that define what insights you should seek for instead of the ones you might think you want.

The COM-B Model is a great example. It proposes that there are three important components to behavior i.e. capability, opportunity and motivation. While this is a great model for ideation and intervention design, you could also use it for user research instrument design as a guide to surface the most salient insights.

In summary, take time to recognise the biases at play in your research process, use this to define and adopt the right mindset as you seek insights and, become tactical with your instrument design by relying on frameworks and models that limit researcher and respondent biases.