Here’s a fact about me. If I get a request to take a survey – be it through email, social media, or the good ol’ fashioned mailbox – I will take it. Whether or not you provide me any incentive to do so (although money and free stuff doesn’t hurt,) I’ll take it because my little researcher heart bleeds for the person on the other end whose job security depends on people like me to provide feedback. I just can’t say no.
Because of this, I have seen my fair share of bad survey questions. Here are some survey no-no’s to watch out for:
This is an amazingly common one and a telltale of poor survey design. See if you can spot the issue:
Should large companies be forced to pay higher taxes?
Personal political opinions aside, this question make its bias clear up front. “Force” is a strong word with a negative use in this context, which could influence how the respondent answers. Take out that leading word, and the question remains neutral:
Should large companies pay higher taxes?
Its so important to spend time with your survey design to avoid a situation in which a respondent is forced to answer in a way that may not reflect his or her personal opinion. For instance:
When you’re out at a party which beverage do you prefer more, liquor or beer?
Well, as someone who often prefers wine in this situation, there’s no way I can truthfully answer this. What if your respondent doesn’t like to go to parties at all? Be wary of the question that tries to force a respondent to answer in a particular way. Keep it simple and try this instead:
What do you like to drink?
DOUBLE BARRELED QUESTIONS
Any survey question that combines issues or subjects is going to present a challenge to the respondent and ultimately skew your results. Take the following example:
How satisfied or unsatisfied were you with your recent hospital visit and the way your doctor explained your treatment options?
What if my doctor explained everything perfectly but my bed was uncomfortable, the nurses were rude, my wait time was long and the food was nasty? Clearly I have varying levels of satisfaction to consider so I can’t answer this question honestly. Keep the subject of your questions separate to ensure accurate data:
How satisfied were you with your recent hospital visit?
How satisfied were you with the way your doctor explained your treatment options?
These are typically yes or no questions that are flagged with absolute statements. Red flag words to watch out for here include “all,” “always,” and “every.” For example:
Do you always cook dinner at home?
The way the question is worded forces a particular response that won’t accurately reflect all the possible outcomes here. A better question would provide options for respondents that could help paint a more thorough picture:
How many days a week do cook at home?
The availability of free and low-cost survey platforms means anyone can be a researcher. These are terrific resources, but without the proper training and a careful eye it is all too easy to overlook some simple but dangerous mistakes that can ruin your data and lead you down the wrong path.
Pay close attention to the questions you ask and when in doubt consult a skilled researcher who can flag the danger zones.
Author Rachel Spencer joined Access in 2007 as part of the PR team, and now serves as Director of Business Intelligence, designing studies, evaluating campaign success and analyzing critical data to inform marketing and PR initiatives on behalf of clients. Click here to learn more.