There are many factors to consider when creating a survey to ensure that you gather the desired information. The way you phrase a question can have a significant impact on the quality of answers and data. You can call it fascinating or simply brain racking. In the end, you can divide survey questions into two categories: closed-ended questions (for quantitative data) or open-ended questions (for qualitative data).
- Were you satisfied with the service? Yes / No (closed-ended)
- On a scale from 1-10, how satisfied were you with the service? (closed-ended)
- What were you most satisfied with? (open-ended)
As you might have noticed, most surveys are based on closed-ended questions that are comprised of multiple-choice, ordinal, interval or ratio questions. Closed-ended questions are easy for survey participants to answer, but obtain a limited set of data, which simplifies survey administration. Because closed-ended questions are easy to answer, they might be the preferred approach for some target groups, depending on the purpose of the survey. Furthermore, the responses can be easily coded and analysed.
However, by focusing on only one type of question, you may not get the full potential of your survey. Qualitative research does not necessarily demand heavy data analysis and data collection does not have to be challenging. Additionally, qualitative research can complement quantitative questions for valuable insight. So, depending on the purpose of your survey, you may want to consider adding open-ended questions.
When open-ended questions can be valuable:
1. To complement closed-ended questions
a) At the end of a group of closed-ended questions or a grid:
Open-ended questions help you gather additional information, allowing you to see the bigger picture. Open-ended questions give you an opportunity to discover something you may have missed. By using open-ended questions in the context to a closed-ended question, you decrease the risk for bias due to misinterpretation. When a question is unclear, survey respondents are more likely to leave a comment in the box provided.
b) As an additional text field to add clarity or premise
Comment boxes or additional text fields provide the opportunity for survey respondents to address certain sub-questions (e.g. in multiple choice questions). These should be optional to answer to avoid unnecessarily extending the survey response time. Keep in mind that the goal is to keep your respondents engaged. Therefore, if you provide options for respondents to elaborate on closed-ended questions, then you increase the chances of gaining extra insight. This approach can also confirm or verify other answers.
2. Free-standing open questions
Sometimes open questions can be very valuable when asked ahead of a closed-ended question or placed as “free-standing” questions. This encourages survey respondents to brainstorm as well as provide spontaneous answers without any influence from earlier options. This type of question can be particularly beneficial for addressing complex issues or for survey evaluation.
Before conducting a series of surveys with a similar design, we recommend being extra cautious and evaluate the questionnaire after first sending it out. Are there any areas you need to clarify or that require more in-depth details? Open-ended questions will help you to avoid mistakes in the long run and ensure that your surveys are successful.
3. Analysis in the word cloud
A word cloud is what it is: a cloud of words, like in the picture shown here. The most common and popular words collected from answers to open-ended questions are shown here. The more certain words are used, the bigger they appear in the word cloud. This is a simplified, visual way of obtaining tangible insights from qualitative data. The data does not necessarily require coding to be valuable; instead, you can use it for reports and presentations. Studies have shown that a word cloud is a simple but powerful graphical visualization of data.
Both types of questions have their place
Is your business improving or standing still? While quantitative questions will give you clear insights in the form of numeric data, open-ended questions will provide you with clear, in-depth insights, such as: What do you need to further develop or improve?
There is no optimal ratio between qualitative and quantitative questions; it often depends on the purpose of the survey and target group. For example, nordic patient surveys have shown that optional comment boxes were appreciated by 76% of the respondents. However, the best approach employs both types of questions in order to see the most value from your research. If you are still stuck and need guidance, learn more in the articles How many questions are too many? or 5 keys to improving survey response rates.