In this guide, we’ll breakdown one of the biggest challenges researchers face when it comes to surveying an audience — Response Bias. Let’s dive in.
You need to know exactly what response bias is, what causes it, and crucially, how to avoid response set bias in your surveys.
In this article, we cover it all:
- What is response bias?
- Why does response bias matter?
- What types of response bias are there?
- How do you get rid of response bias?
- Which survey tools should your company use?
What Is Response Bias?
This term refers to the various conditions and biases that can influence survey responses. The bias can be intentional or accidental, but with biased responses, survey data becomes less useful as it is inaccurate.
This can become a particular issue with self-reporting participant surveys.
Bias response is central to any survey, because it dictates the quality of the data, and avoiding bias really is essential if you want meaningful survey responses.
Leading bias is one of the more common types. An example would be if your question asks about customer satisfaction, and the options given are Very Satisfied, Satisfied and Dissatisfied. In this instance there is bias that can affect results.
To avoid bias here, you could balance the survey questions by including two of each of the positive and negative options.
Actionable insights from 1,000+ professionals.
Why Does Response Bias Matter?
A survey is a powerful tool for businesses because it provides the ability to obtain data and opinions from real members of the target audience, which gives a more accurate assessment of market position and performance than any trial-and-error tests could ever produce.
When the goal of the survey is data collection, having the right sample size and survey methodology matter most.
What Types of Response Bias Are There?
One of the key things to avoid response bias is to fully understand how it happens. There are several types of response bias that can affect your surveys, and the ability to recognize each one can help you avoid bias in your surveys as you create them, rather than spotting it later.
However, even with this understanding, it is always wise to have several people go through any survey design to check for possible causes of response bias before any survey is sent to respondents. This ensures that the resulting data is as accurate as possible.
We will cover the main types of response bias here, and we will provide examples of response bias to show just how easy it is to introduce bias within the survey.
1) Demand Characteristics
One of the more common types of response bias, demand bias, comes from the respondents being influenced simply by being part of the study.
This happens as respondents actually change their behavior and opinions as a result of taking part in the study itself. This can happen for several reasons, but it is important to understand each to be able to deal with each form of this response bias problem.
Participants who look to understand the purpose of the survey
For instance, if the survey is looking into a user experience of a website, a participant may see that the aim is to gather data to support making changes to layout or content.
The participant may then answer in a way that supports those changes, rather than as they really think, resulting in untruthful or inaccurate responses.
The setting of the survey or study
This is more applicable to surveys carried in person, where researchers conducting the survey can have an influence on the respondents, but it can apply to digital surveys too.
Interaction between researcher and respondent
This can influence how the survey is approached. Note that if it’s a digital survey that researcher-to-respondent interaction is still possible, occurring in the email or message used to invite the respondent to participate.
Wording bias can come into effect here as well
This type of bias influences the entire gamut of responses from individual or multiple participants. For instance, if the researcher knows the participant personally, even greeting them in a friendly manner can have a subconscious effect on the responses.
This is as true in an email as it is in person, so by retaining a formal approach to all participants regardless of who they are you can ensure a uniform response from all participants.
Prior knowledge of the survey
Whether the questions themselves, or the general aims of the survey, or how it is being put together, prior knowledge of some aspects of the survey deliver response bias.
This is because the participants can become preoccupied with the survey itself, resulting in those participants second-guessing their own answers and providing inaccurate responses as a result.
2) Social Desirability Bias
This type of response bias results from participants answering sensitive questions with socially desirable, rather than truthful answers.
The key here is how response bias questions are worded. To better illustrate this, here is an example:
“Do you think it’s OK to drink alcohol frequently?” is a question that society conditions us to say no to, even those who do drink heavily.
A question like this cannot get a truthful response.
People know the appropriate answer and will give it regardless of their real view.
In fact, social desirability bias can work both ways. Depending on the question you can find both over-reporting and under-reporting of a particular viewpoint, and that can have a dramatic affect on the usability of the data eventually generated.
3) Extreme Responses
The name gives this one a way really, but it refers to the survey bias that is characterized by respondents providing extreme answers to questions. This can manifest as extreme positive or negative responses, and both render the data ineffective.
This type of bias occurs most commonly in surveys that offer a scale for responses in order to rate individual components, whether that is numbers (such as 1 to 5, star ratings) or even a selection of statements (such as satisfied, mostly satisfied, somewhat dissatisfied, and dissatisfied).
What happens is that the answers tend to be either 1 or 5, 1 star or 5 stars, satisfied or dissatisfied. Few people choose the middle options. Through various response bias studies, it has been shown that there are cultural influences on this kind of behavior.
Culture is of course not the only cause of this behavior. Studies show that education level also matters. The lower the education level, the more likely the participant display extreme response traits.
Lastly, wording bias can also be a cause. Sensitive questions that offer blame to someone or something for a difficult situation will result in extreme responses too.
Here’s an example
Think about the following question:
“Are family and peers partly responsible for a substance abuser’s behavior?”
There is no middle ground there for most people, and responses will be at the extreme end of whatever options are provided. While sometimes sensitive questions need to be asked, carefully structuring them to avoid the instant emotional response can avoid the extreme responses we are talking about.
4) Neutral Responding
This type of response bias is the exact opposite of extreme responding, as here the participant chooses the neutral answer every time. This is usually a result of the participant not being interested in the survey at all and is simply looking to answer questions as quickly as possible.
This inaccurate data is just as damaging and highlights just how important selection of participants can be for your research.
5) Acquiescence Bias
Acquiescence bias is a form of response bias where participants respond in agreement with all questions within the survey. In most cases, if your survey is well designed, that results in the participant agreeing with at least two contradictory statements. The answers provided this way are then no longer accurate or truthful.
The reality is that all of us have our unique view of the world, and it is highly unlikely anyone will agree with everything on a survey. To figure out how to combat this type of bias, let’s look first at how this kind of bias occurs.
There has been much research carried out on this kind of response. American educational psychologist Lee Cronbach theorized that it is a result of the participant looking at the question and actively searching for information from their own experiences to support it. Others have suggested it stems from a need to please researchers, but Cronbach’s approach is considered the most relevant.
As an example of this, a survey may contain two questions:
- “Are you outgoing and social?”
- “Which do you prefer: a quiet night in or a raging party?”
A respondent could answer yes to the first question, because we all like to think of ourselves as likeable, social people.
However, when faced with the second question, they may remember a nice night in they had enjoyed recently and answer with that. Those two answers may seem to contradict each other. Both, however, would be genuine, truthful answers from the participant’s perspective.
6) Dissent Bias
Finally, we have dissent bias. This is the exact opposite of acquiescence bias where the participant seeks to disagree with every statement or question the researcher makes.
With the same causes, dissent bias can be just as problematic for researchers as acquiescence bias, and it must be remedied in the same way.
How Do You Get Rid of Response Bias?
Avoiding response bias is easier when you know the types of response bias, and why they occur. There are several things that every researcher can do to overcome response bias.
1) Understand Your Demographic
As we discussed previously, some demographics are more susceptible to certain types of bias. Paying attention to the ‘who’ you are asking is as important as ‘what’ you are asking.
You can put demographic understanding at the top of your research by asking yourself a few simple questions:
- What does my audience have in common?
- Why will they want to answer my survey?
- What is it about them I am interested in?
Make these part of your survey research and you’ll have a higher response rate.
2) Avoid Question Wording Bias
Careful question phrasing is critical to get an unbiased survey and responses. We talked before about inherent bias, but this also applies to questions that produce an emotive response and can cause acquiescence bias or extreme responding.
Follow these simple survey best practices to avoid word bias in your surveys:
- Look at the questions and ensure there is a balance
- Include as many positive responses as negatives
- Seek to avoid questions that will produce a powerful emotional response one way or the other
- Always look for phrasing that provides acceptable positive or negative answers
3) Diversify Questions
To keep participants focused, avoid using one type of question all the way through. Instead, mix binary response questions (yes or no) with those that offer a range of responses. This avoids the participant simply giving the same answer each time, and thus forces them to think about their responses.
This applies to more than just the style of question. Avoid asking questions about the same topic one after the other, as this can lead to reflex answers given without thought.
Instead, mix up topics throughout the survey, so that the participant is not just automatically following what they answered last time and has to think about each question on its own.
4) Allow Participants to Say ‘No’
Participants often wish to give the best answer possible, but for some questions a participant may not have an answer. Feeling that they need to do something, they at times will give an inaccurate answer that you don’t want.
Having a ‘don’t know’ or ‘undecided’ option allows them to answer honestly without skewing your results. In fact, this minor addition to your survey could remove a lot more inaccurate answers than you think.
In addition, the ‘don’t know’ option can be a valuable data set of its own and can prove useful throughout subsequent analysis.
5) Effective Administration
As we have seen, several forms of response bias can be caused by participants being influenced by the researchers themselves, or the motives behind the survey. This makes every aspect of the administration of the survey crucial to its success.
The following are important steps every survey should follow:
Researchers must remain neutral at all times
During all correspondence of any kind, maintain a professional, unbiased demeanor to ensure participants recognize the importance of the situation. Using the digital approach can remove some of these risks.
Maintain survey integrity
Participants second guessing research motives or finding out motives before taking the survey both result in response bias. Do this by maintaining the integrity of the survey and ensuring participants do not have additional information.
To ascertain if participants have any understanding of the survey motives, a short after-survey questionnaire can be used.
Nextiva’s 2020 State of Business Communication Report.
6) Avoid Emotionally Charged Terms
Your questions should be clear, precise, and easily understandable. That means simple, unbiased language that avoids using words that evoke an emotional (rather than through-based) response.
This includes words such as:
- Politician
- Terrorist
- Junkie
And so on. You want answers that are thought through, and these so-called ‘lightening’ words instead elicit an emotional response that is not as valuable for your research.
In addition, try and avoid using a lot of negative words, as these can have an effect on how the participant perceives the survey and skew their responses.
Finally, avoid word tricks. Be transparent with the audience and allow them to develop their answers.
All of these are relatively simple steps to deliver improved survey results. Removing response bias will help you acquire accurate, unbiased data, and reflect the real views of your audience.
What Survey Tools Should Your Company Use?
Even if you know the various types of response bias, you still need to monitor the survey for problems and inaccurate data.
Nextiva has two tools designed to do this for you, providing performance combined with ease of use for seamless integration into your workflow.
Nextiva Survey Analytics
Survey analytics provides business intelligence efficiency with a comprehensive feature set that tracks survey response data throughout your research. This provides simple, clear, visual presentation of the data you need.
Through this simple interface you can drill down to get a complete performance picture, analyzing results right down to the individual respondent if necessary. All data is easily accessible, saving time and frustration.
With a visual representation of aggregate responses to any question, you can quickly identify trends and anomalies as they are occurring.
Nextiva Surveys
A complete software solution for all your surveys, Nextiva Surveys provides the perfect platform for all your research. With a simple, fast design solution, your surveys will look great. And full customization ensures they always reflect your brand image.
With no coding required, you can get beautiful, rich surveys put together in just a few minutes, saving time and money without sacrificing quality or control.
There are templates for all types of questions, complete security, and features including skip logic for a personalized experience.
Nextiva Surveys has all the tools you need to create response bias-free, effective surveys that produce exceptional results.
Platform free, you can deliver surveys in an email or via the web. The responsive design provides an exceptional user experience even on mobile devices.
When it comes to response bias-free surveys, Nextiva has your back. With our easy-to-use survey software creating effective surveys that avoid response bias are just a few clicks away. Try them now and see how easy surveys can be!
