There are many factors to consider when creating a survey to ensure that you gather the desired information. The way you phrase a question can have a significant impact on the quality of answers and data. You can call it fascinating or simply brain-racking. In the end, you can divide survey questions into two categories: closed-ended questions (for quantitative data) or open-ended questions (for qualitative data).
Examples of open-ended and close-ended questions:
- Were you satisfied with the service? Yes / No (closed-ended)
- On a scale from 1-10, how satisfied were you with the service? (closed-ended)
- What were you most satisfied with? (open-ended)
As you might have noticed, most surveys are based on closed-ended questions that are comprised of nominal, ordinal, or interval questions. Closed-ended questions are easy for survey participants to answer, but obtain a limited set of data, which simplifies survey administration. Because closed-ended questions are easy to answer, they might be the preferred approach for some target groups, depending on the purpose of the survey. Furthermore, the responses can be easily coded and analysed.
Overview of measurement scales:
Nominal: A simple multiple choice that presents variables in different classifications but without a quantitative value or order. For instance, choose between answer A or B.
Ordinal: An ordinal scale, for instance, is used to describe non-mathematical ideas, like satisfaction, that can be represented with emoticons for instance. You can see it in airports or facilities, where you can indicate with a smiley face on a screen how happy you were with the service.
Interval: On an interval scale, you could rate your experience in numerical values from say 1 to 10.
While these do provide you with valuable insights, by focusing on only one type of question, you may not get the full potential of your survey. The answer options can give you an overview, but close-ended questions will not help you go into the depths of the reasons for the actual, personal experience. They do not demand explanation or reflection even. By adding open-ended questions you can tackle those blind spots.
Qualitative research does not necessarily demand heavy data analysis and data collection does not have to be challenging. Additionally, qualitative research can complement quantitative questions for valuable insight. So, depending on the purpose of your survey, you may want to consider adding open-ended questions.
When open-ended questions can be valuable:
Examples of open-ended questions for qualitative research can come in many forms: as a stand-alone questions, as a follow-up to a close-ended question, as comment sections, and many more. Let’s have a look at a couple of these and in which cases they are valuable.
1. To complement closed-ended questions
a) At the end of a group of closed-ended questions or a grid:
Open-ended questions help you gather additional information, allowing you to see the bigger picture. Open-ended questions give you an opportunity to discover something you may have missed. By using open-ended questions in the context to a closed-ended question, you decrease the risk for bias due to misinterpretation. When a question is unclear, survey respondents are more likely to leave a comment in the box provided.
b) As an additional text field to add clarity or premise
Comment boxes or additional text fields provide the opportunity for survey respondents to address certain sub-questions (e.g. in multiple choice questions). These should be optional to answer to avoid unnecessarily extending the survey response time. Keep in mind that the goal is to keep your respondents engaged. Therefore, if you provide options for respondents to elaborate on closed-ended questions, then you increase the chances of gaining extra insight. This approach can also confirm or verify other answers.
2. Free-standing open questions
Sometimes open questions can be very valuable when asked ahead of a closed-ended question or placed as “free-standing” questions. This encourages survey respondents to brainstorm as well as provide spontaneous answers without any influence from earlier options. This type of question can be particularly beneficial for addressing complex issues or for survey evaluation.
Before conducting a series of surveys with a similar design, we recommend being extra cautious and evaluate the questionnaire after first sending it out. Are there any areas you need to clarify or that require more in-depth details? Open-ended questions will help you to avoid mistakes in the long run and ensure that your surveys are successful.
3. Analysis in the word cloud
A word cloud is a name given to a visual arrangement of words on the page that corresponds to their importance (i.e. often they were used in your answers), as shown in the graphic below. The most common and popular words collected from answers to open-ended questions are shown here. The more certain words are used, the bigger they appear in the word cloud. This is a simple, visual way of obtaining tangible insights from qualitative data. The data does not necessarily require coding to be valuable; instead, you can use it for reports and presentations. Studies have shown that a word cloud is a simple but powerful graphical visualisation of data.
Both types of questions have their place
Is your business improving or standing still? While quantitative questions will give you clear insights in the form of numeric data, analysing open-ended questions will provide you with clear, in-depth insights, such as What do you need to further develop or improve?
There is no optimal ratio between qualitative and quantitative questions; it often depends on the purpose of the survey and target group. For example, nordic patient surveys have shown that optional comment boxes were appreciated by 76% of the respondents. However, the best approach employs both types of questions in order to see the most value from your research. If you are still stuck and need guidance, learn more in the articles How many questions are too many? or 5 keys to improving survey response rates.