How do I word survey questions ?
Questions can either be Yes/No, or you can ask people to give a rating to indicate strength of feeling about an issue. If you want to elicit a rating, there are two ways of putting this. You can ask for feedback either by asking people to reply to a question, or you can ask them to state whether they agree or disagree with a statement. Below are examples of each of these, along with some tips for minimising customer mistakes.
1. Statement wording
“In a moment I am going to read out a list of statements. Please record the extent to which you agree or disagree with each statement by using any key on your phone, between 1 and 4. Press 1 if you strongly disagree with the statement made, through to 4 if you strongly agree with the statement.”
Q1: I felt the person who took my call was pleasant and polite.
Q2: I felt the person who took my call clearly understood my query or concern?
Confirmations:
1: Strongly disagree
2: Disagree
3: Agree
4: Strongly agree
Advantages of statement wording: Instructions are consistent and simple. Questions can be asked in any form, the confirmation is always the same. It is simple to word statements and confirmations. Linguistic / grammatical problems are rare, the wording tends to work for any issue.
Disadvantages: People may get confused about the scale and think that they are giving it a position rather than points (i.e. one is first).
2. Question wording
“In a moment I am going to ask you to give us marks out of 5 for various things to do with the way we dealt with your call today. Please press 5 if you think we did very well through to 1 if we did not do very well. Remember 5 is the highest mark and 1 is the lowest mark. I will now ask the questions.”
Q1: How pleasant and polite was the person who took your call?
Q2: How well did the person who took your call understand your query or concern?
Possible confirmations
1: That’s the lowest / 1 out of 5 – Poor
2: That’s a low mark / 2 out of 5 – Not satisfactory
3: That’s a medium mark / 3 out of 5 - Ok
4: That’s a high mark / 4 out of 5 – Good
5: That’s the highest mark / 5 out of 5 – Very good
Advantages of question wording: People tend to understand the scale more easily and confirmations are more distinctive
Disadvantages of question wording: It can be harder to construct linguistically, both as a question and the response confirmations. E.g. Would you recommend this service to a friend? A rating of 3 would require a confirmation of something like “3 – neither recommend or not recommend”, which sounds awkward.
Minimising Mistakes
The main source of mistakes is people getting the range the wrong way round. If you have a short range (e.g. 1-4), you can reiterate each point on the range in the question itself. If you have a long range (e.g. 0-9), you can speak the extreme points on it, e.g. emphasise that high is good and low is bad. In either case you should probably only speak or remind the customer of the scale for the first few questions. Repeating it after 3 questions is probably unnecessary and lengthens the survey.
You also need to give people a chance to correct mistakes. This can be done by using the ‘Hash key’ to go back to a previous question. The only disadvantage is that you lose control over the length of the call, and therefore the cost of it. Evidence shows however that few people tend to use this facility, so you might want to specifically encourage people to go back and correct any mistakes.
No survey will be 100% accurate. However, if you are primarily interested in trends, i.e. changes over time, the level of mistakes does not particularly matter, as there is no reason to assume the proportion of people making mistakes will vary over time, and so it should not affect the trends.