Kevin Godby

My presentation this week is a short collection of pet peeves and tips for writing survey questions.

1. Make answers mutually exclusive.

   How many days per week do you wear socks?  0–1  1–2  2–3  3–4  4–5	How many days per week do you wear socks?  0–1  2–3  4–5  6–7
   BAD	GOOD

   If I wear socks three days a week, I don’t know which option I should select in the first example. In the second example, the choice is obvious.

2. Don’t make them reach for their calculators.

   How many hours a year do you use the Internet?	How many hours a day do you use the Internet? How many hours a week do you use the Internet?
   BAD	GOOD

   In the first example, I would have to estimate how many hours a day I spend on the Internet and then multiply that a few times to get the hours per year. That’s a lot of work! Instead, you should estimate what the proper time period should be when writing the question. If you anticipate that people use the Internet a few hours each day, then ask for the answer in hours per day. If you suspect that they only use the Internet a few minutes each day or a few hours each week, change the wording accordingly.

3. Provide a neutral option on the Likert scale.

   Cold pizza is good for breakfast.  Strongly disagree  Disagree  Agree  Strongly agree	Cold pizza is good for breakfast.  Strongly disagree  Disagree  Neither agree or disagree  Agree  Strongly agree
   BAD	GOOD

4. Make it clean when the respondent can provide multiple answers.

   Which flavor of ice cream do you like?  Vanilla  chocolate  Strawberry Which pizza toppings do you like?  Pepperoni  Cheese  Sausage  Anchovies	Which flavor of ice cream do you like? (Choose one.)  Vanilla  chocolate  Strawberry Which pizza toppings do you like? (Choose all that apply.)  Pepperoni  Cheese  Sausage  Anchovies
   BAD	GOOD

   It should be obvious to the respondent when they can select more than one answer. In the second example, I’ve added the notes “(Choose one.)” and “(Choose all that apply.)”. I’ve added another small visual clue by changing the selection circles to boxes to indicate that more than one choose is allowed.

5. Don’t provide so many options that it’s difficult to choose.

   What is your favorite color?
   Amaranth  Amber  Amethyst  Apricot  Aqua  Azure  Beige  Black	Blue  Brown  Cerulean  Cinnamon  Copper  Coral  Cream  Crimson	Cyan  Emerald  Eggplant  Forest green  Fuchsia  Gold  Goldenrod  Grey	Green  Indigo  Ivory  Jade  Lavender  Lemon  Lilac  Lime

   This one is pretty self-explanatory. If you overwhelm the respondent with choices they’ll likely give up and pick the first one that seems reasonable.

6. Don’t exclude possible responses.

   How many hours a week do you spend watching television?  1–5  6–10  11–15  16–20	How many hours a week do you spend watching television?  none  1–5  6–10  11–15  16–20  more than 20
   BAD	GOOD

   You should consider the edge cases. For instance, the respondent may never participate in an activity or perform some task. Also, you shouldn’t put an artificial upper limit on the responses.

7. Placing the demographic questions at the end of the survey may result in more complete responses since the respondent has already invested time in the survey.
8. Instead of asking “How old are you?”, ask “In what year were you born?“ You’ll often get more responses this way.
9. If you’re asking an open-ended question and provide lines on which to write the answer, double-space the lines so the respondent has plenty of room to write. (I hate it when I have to squeeze my handwriting into a 1/8-inch high space.)
10. Clearly indicate which questions are required and which are optional.
11. When requiring personal information, explain what you will use it for.

Do you have any pet peeves or tips regarding survey questions? If so, I’d love to hear them; leave a comment below!

References and Resources

• Kevin’s presentation slides (PDF)
• Kevin’s presentation video (Quicktime movie)