It may be necessary, in particular to collect the needs and opinions of users, to design a questionnaire. It allows the collection of subjective data relating to the opinions, satisfaction, beliefs, self-description, knowledge, behaviours, uses or intentions of given users (a sample), chosen as being representative of the overall population from which they have been extracted, and for which one wants to have a certain amount of information.
Preparation
The design of a questionnaire includes the implementation of the following steps :
- Define and select a sample of people to be interviewed.
- Define what information you want to collect through this questionnaire, and if so, what assumptions you want to test.
- Design questions: wording, order, closed questions, open questions, etc.
- Implement questionnaire delivery (paper version distributed/collected or online)
- Organize the data entry and process the results: flat sorting, cross sorting, descriptive and inferential statistics.
Sampling allows us to define who we want to interview, i.e. the target population. More often than not, and on small samples, it will be sufficient to take randomly within this target population, considered as homogeneous (for example, all the users of such or such interface have the common point of being users). However, an empirical sampling method based on known or assumed quotas may be used. This will try to ensure that the sample is representative of the characteristics of the mother population: for example, age, sex, family situation, place of residence, occupation, income level.
Design
Once the preliminary scoping work has determined the type of information and data to be collected, the questionnaire itself must be developed. A certain number of rules must be respected:
- Structure the questionnaire starting with the simplest and general questions and ending with the most complicated and personal ones.
- Limit the number of questions, and limit yourself to useful questions.
- Ask short and easily understandable questions (avoid ambivalence). Avoid double negations, passive verbal forms, make sure that the terms used are known to the target audience. The emotional value of the words must be taken into account, the wording of the questions should not be connoted.
- Choose the types of questions according to the goals pursued and the analysis that one wants to make of them afterwards:
- Closed-ended questions (collecting answers is simple and reliable: simply tick the chosen answer. However, this simplification effort may be simplistic, and may not allow the collection of finer nuances in the expression of responses).
- Single choice (ex: age, sex, etc.).
- Multiple choice.
- Open-ended questions (informative, but difficult to search and process as they require post-coding. This consists in grouping the answers given to an open-ended question into categories that are as homogeneous as possible. The exercise is difficult, subjective and tedious. Automatic methods can be complicated and expensive to implement).
- “Scale” question (Lykert scale type, measure an opinion on a set of points distributed between two extreme values e.g. from “strongly agree” to “strongly disagree”). Where appropriate and depending on the objectives pursued, odd or even scales should be chosen to allow (or not) respondents to express a neutral or median position in their responses (odd scales).
- “Control” questions (this type of question is only used to ensure that respondents complete the questionnaire seriously. The control question(s) can, for example, test the consistency of the proposed answers in order to highlight “lax” people in their reading of the questions.
Analysis
Once the results have been collected, they must be processed. This begins with “cleaning” the data: detection of outliers, ceiling or floor effects (concentration of all responses on an extreme value) and recoding of variables (especially in the context of processing open-ended questions). Then comes the actual data processing: flat sorting (flat sorting makes it possible to obtain the distribution of responses and takes the form of “frequency tables”). and cross sorting (Cross sorting is a multivariate extension of flat sorting. They allow to cross the data of different flat sortings, for example the crossing of answers and profile information. It is a linking of results to several questions.). In the latter case, it is possible to carry out more elaborate inferential statistical treatments, which make it possible to determine if there are “effects” in the way people answer the questions :
- Distribution of the answers at random or not. Groups of related questions (correlation, factor analysis).
- Groups of respondents (discriminant analysis).
- Differences between groups (analysis of variance, parametric or not).
- Prediction of responses to one question from another (regressions).
One example is the KHI 2 test, carried out since the beginning of the 20th century, which aims to measure changes in variable distributions in different populations in order to check whether this difference is statistically significant.