Questionnaire Questions

Many people find themselves in a position where they know they could benefit from conducting a questionnaire but have difficultly on deciding what questions to ask.

Since questionnaires come in a variety of shapes and sizes and can be conducted for a plethora of reasons there is no definitive answer to what the right questionnaire questions should be.

However by following some simple guidelines and avoiding the wrong type of questions it is possible to devise a questionnaire full of the right type of questions.

The Questionnaire Objectives

To start with decide from the very beginning what the objective and purpose of the questionnaire is. What are the triggers for conducting a questionnaire, how will the results be analysed and what are the expected benefits?

The specific goals will vary depending on the type of questionnaire, for example the objective of an employee questionnaire might be to record on a regular basis the levels of employee satisfaction, or might be triggered due to a period of change. The purpose of a customer questionnaire might be to ensure that service levels are operating to a high standard or it might be trying to gauge the support for a proposed new service.

Questionnaires can be used in commercial and academic research, service level assessments, marketing, educational, customer, employee, health & safety and record keeping, and many more to boot, but no matter the type of questionnaire they will all have a purpose, otherwise they are not worth doing.

Target Respondents

Decide who you are going to ask? For Employee or Customer questionnaires this will be relatively simple, for questionnaires that are related to market and academic research or marketing, a target audience may not be obvious.

Consider how many respondents you require to achieve meaningful results. More does not necessary equate to better. If the questionnaire is on a subject of something that is very niche it will be important to screen out those people canvassed that may not be sufficiently qualified to answer.

Essential and Non-Essential Questions

With the objective of the questionnaire in mind, brainstorm the questions that should be asked and split the questions into two categories, that we will call 'Intelligence' and 'Analysis' questions.

• Intelligence Questions will be those questions where the answer will gather insightful information that will help achieve the questionnaire's purpose.
• Analysis Questions will be those questions where the answers will help analyse the results for example, age, gender, length of service.

Take all the questions in Intelligence Question category and divide the proposed questions into two piles 'Need to know' and 'Nice to know' questions.

A 'Need to know' question will be a question that is considered essential in helping to achieve the questionnaire's objective.

A 'Nice to know' question is one that may not directly relate to the objective, but could be perceived as being useful information. For example, asking for someone's marital status, it may have no bearing on the immediate objective but may represent an opportunity to gather information for other purposes. We will want to keep 'nice to know' questions to a minimum, if the questionnaire is relatively short, or offers those participating a big incentive to complete the questionnaire then some 'nice to know' questions can be included.

Types of Questions

Questions general fall into one of two categories, 'open' and 'closed'.

An open question is one that allows the respondent free reign to answer in their own words. For example, 'What are the areas of your job that you are most enjoy?'.

A closed question is one where they are given predefined answers, for example ' On the whole please indicate how much do you enjoy your job?": A lot, Okay, Not a lot.

Open questions have the advantage that the response can encourage rich qualitative data and freedom of expression, on the negative side the responses are more difficult to analyse can be considered time consuming to answer and may disadvantage those who are less literate than others.

Closed questions have the advantage that they generate quantitative data that is easier to analyse, are quick to answer and are less challenging for those who are less literate, they can however encourage less thoughtful responses.

It is generally recommended that a question is kept closed unless it justifies being made open, but keep in mind the more open questions the more difficult and time consuming and subjective the analysis.

Phrasing of the Questions

To suggest that not all questionnaires are designed to be impartial is perhaps a little controversial. It is however a fact that questionnaires are often used to gain support for a particular argument and are therefore less concerned with being unbiased and more focussed on achieving a positive result.

However, if a questionnaire's questions are blatantly biased then opponents will have no problem discrediting a questionnaire's findings and therefore remove any potential benefit.

If questionnaires were prepared by two groups, one supporting and one opposing a new airport runway, the group supporting the new runway may distribute a questionnaire across a wide area where the questions concentrates on the economic benefits that a new runway might bring, while the opposing group's questionnaire is likely to focus on the local community and the environmental aspects.

It is a fact that questionnaires can be used to generate statistics that support an argument that sound impressive but may not pass scrutiny.

Where there the intention is for the questionnaire to be unbiased the following principles can be applied.

Avoid:

• Leading the respondent, such as 'Is it right to say .', 'Would you agree ..'
• Bias, such as 'Do you think that irresponsible drivers who speed should be fined?'
• Subjective terms such as 'often', 'regularly', 'local' as these terms mean different things to different people; if they are used qualify the questionnaire's meaning such as 'often, 3 or 4 times a day'.
• Abbreviations, jargon and colloquialisms unless you are confident that the target respondents will all fully understand the question.

  For example if the questionnaire is aimed at qualified doctors, technical jargon will probably be justified, if the questionnaire is aimed at internal personnel, not all jargon and abbreviations may be known to everyone. If colloquialisms are used it may disadvantage those that are not native speakers of the language used in the questionnaire.

• Asking double barrelled questions like 'Do you like tennis and golf?' or 'Did you find the trainer both confident and knowledgeable'; instead ask one question at a time.
• Double negatives such as 'Should those with five years or more service not be entitled to extra holiday?'; instead rephrase the question 'Should those with five years be entitled to extra holiday?'.

Ordering the Questions

If possible break any questionnaire with more than a half dozen questions into clear sections.

Consider asking the most important questions first so that if the respondent fails to complete the questionnaire some useful intelligence may still be obtained.

Balanced this with the need to ease respondents into a questionnaire by not starting with difficult, awkward or embarrassing questions.

As a guide, start with some easy and general questions then move from 'General to particular' and 'Factual to Abstract'.

For each section and where it is feasible to do so start with closed questions and finish with open questions.

Answer Options

• For closed questions ensure that the answer options allow for all eventualities, consider including a 'get out of jail' answer option such as 'Not applicable', 'Rather not say', 'Don't know', it is better to have a positive non-answer than a missed or a forced answer.
• As a safeguard, but only if thought necessary, include the 'Other' facility allowing a free text response to a closed question.
• Avoid answer options that overlap, such as age groups 20-25, 25-30, 30-35, instead use groups that are distinct from one another such as 20-25, 26-30, 31-35.
• Where possible use proper headings rather than a scale that is equated to headings, i.e. instead of ' Rate how much you agree or disagree to the following statements where 1 is totally disagree and 10 is totally agree'; instead use 'Totally agree', 'Agree', 'Disagree' etc as the answer options.
• If using scales do not use scales that are too broad, such as from 1 to 100, or even 1-10; Unless there is a good argument for doing otherwise the analysis will normally be grouped into two levels above and two levels below the medium - with the possibility of having a 'no answer' option. If that is what the analysis will produce, save time by having the answers grouped from the very beginning.
• Consider the benefits of randomly ordering the answer options to minimize order bias, where the same answer option is positioned in the same place for each question.

Analysis Questions

Consider each question in turn and ask if it will be sufficient to analyse the information in isolation or would the information need to be broken down using other metrics such as gender, age, department etc.

Consider if any screening questions are required so that 'unqualified' responses can be eliminated from the results.

If there is a potential requirement for the information to be broken down, ensure that the metrics required for the analysis are included in the questionnaire.

Once the questionnaire is substantially complete double check that each question is 'on message' with the questionnaires overall objective. If it is thought that the questionnaire can accommodate 'nice to know' questions then include them but only in moderation.

For more information or to discuss how online questionnaires can help you please contact surveys@surveygalaxy.com or visit www.surveygalaxy.com the quick, easy and cost effective way to do online questionnaires.