In a market research study, the way questions are asked sets the trajectory for everything else to come. Poor questions lead to poor results, and if the data is bad, not much can be rectified. Analyses can be rerun, and reports can be rewritten, but once a question has been asked, its influence cannot be taken away.
This can feel like a lot of pressure. And you won’t always have the time or resources to come to a market research firm like us for help. So, we want to give you some tips to consider when formulating your own questionnaires.
While there are many factors that go into asking the right questions, we’re starting with the ones we see time-and-time again that trip people up.
Remember, accessible doesn’t mean simple.
It’s important to take time to make sure that all the questions are easy to understand. This may mean writing questions at a middle school reading level. But this doesn’t mean writing questions using the simplest language possible.
For example, say the question asks:
“How easy or difficult was it to comprehend the instructions provided at your most recent appointment regarding when to take your medication?”
There are a lot of words here that bury the leading question and will likely cause confusion and maybe even deter a participant from completing the questionnaire.
But if you simplified the question to be:
“How easy or difficult was it to understand your care instructions?”
You’re missing the specific focus on medication and may get answers unrelated to what this particular study is trying to learn.
Simplicity often comes at the loss of specificity. Therefore, it’s your responsibility to find a balance between using specific concepts related to your study (defining and contextualizing as needed) and ensuring that the overall language is largely universal.
Instead, try this approach:
“At your last appointment, how easy or difficult was it to understand the instructions about when to take your medicine?”
This question gets you the information you need without confusing the participant or losing the nuance of what you are specifically trying to learn.
Using tools to check the reading level of a research instrument and having multiple people (for us it’s at least 3) read through your questions before conducting the research are a couple ways you can ensure you’re asking questions that are truly accessible.
Beware of leading questions.
Preventing bias goes far beyond being understood. Even if a respondent can clearly understand the words and concepts, they may still have a difficult time expressing their honest opinion depending on how you’ve worded the question.
Take this survey question for example:
How much do you agree that Brand X is a good brand?
1 – Strongly agree
2 – Somewhat agree
3 – Neutral
4 – Somewhat disagree
5 – Strongly disagree
6 – Prefer not to respond
Now that you’ve read the question, you’re thinking about Brand X as good, even if that may not be how you felt initially. Even though there are response options to express disagreement, you’re still subconsciously thinking about Brand X in a positive light. That is positivity bias.
And succumbing to it is normal. We are all easily influenced and many of us have the desire to please. Unless we have strong negative feelings, most people generally like being agreeable and want to go along with what they perceive to be the answer you’re looking for.
A less biased way of asking a question like this would be:
“Which of the following best describes your feelings toward Brand X?”
And if it were open ended:
“How would you describe your feelings toward Brand X?”
Avoid response options that bias results.
While a leading question can bias results, so can the response options. When trying to uncover respondents’ perspectives on a certain topic, it is important to make sure that they have balanced response options that include an equal number of negative and positive choices; particularly when respondents can only answer from the choices they are given (i.e. there’s no specify other).
Take our revised question above:
Which of the following best describes your feelings toward Brand X?
It’s not unusual for us to see scales like the following:
1 – Very positive
2 – Somewhat positive
3 – Positive
4 – Somewhat negative
5 – Very negative
This list of response options presents 3 “positive” options and 2 “negative” and thus fails to present a balanced scale. Respondents who have a neutral attitude toward the brand will be forced into “positive” or “somewhat negative”, skewing the findings. Furthermore, if all “positive” findings are aggregated, it will likely overstate the actual positive sentiment toward the brand.
This can be addressed with a simple fix: switching your middle option from “Positive” to “Neither positive nor negative.” It achieves a level of neutrality that can elicit honest and unbiased responses from participants.
Remember, the results of your survey will only be as high-quality as the response options given to participants.
Order questions with intention.
Consumers are still thinking of the questions they just answered when answering a new set of questions. Because of this, it is important to order questions in such a way that they do not bias the ones that follow.
For example, if respondents are asked to talk about a negative experience they’ve had with a brand, and that question is then followed by asking them to rate that same brand, it could skew their rating in a negative direction.
Another instance might be asking a top-of-mind recall question too late in the conversation or survey, after already prompting the respondent to think about a specific brand by including it in prior questions. Generally, we believe questions should funnel down in order of most broad to most specific.
After all of this, you may be wondering whether you really need to know this when AI exists to help you generate questions.
Our short answer: you really do.
While AI can feel accessible and convenient to lean on, it often breaks the rules and gets things wrong, even when you explain your research objectives clearly.
That’s why it’s critical to bring in your discernment to question writing. You need these tools in your back pocket whether you’re writing your own questions or vetting suggested questions you’re given, from AI, your client, or otherwise.
With these pointers in mind, you will set yourself and your project up for success in uncovering insights that will actually inform your strategic goals.