What is the right number of screener questions to ask? How many screener questions is enough? How many is too many?The purpose of a screening questionnaire, or screener, is to screen for criteria you want, or don’t want in your research participants.
The number of screening questions is an important consideration. Too many screening questions and potential participants are likely to lose interest. Too few screening questions and you won’t get enough information to properly qualify folks.
What’s an average number of screener questions?
To answer this question we analyzed data from 2705 projects posted on the Respondent platform.
According to our researchers, who include design, research, marketing, and product management experts from companies like Airbnb, Uber, P&G, IBM, SAP & Microsoft, the average number of screener questions asked per recruitment project is 9.8.
So let’s round up to 10.
However, this doesn’t reflect the range of screener questions asked on the platform, or the target audience, or the type of study. So we broke down this number a bit more.
What’s a normal range of screening questions?
As you can see from the box and whisker plot, 50% of projects posted on the Respondent platform have between 5 and 12 screener questions, with the median being 8 screener questions.
So as long as you have more than 5 and less than 12 screener questions, you’re going to be in the majority of researchers out there. [Shout out the researcher with 74 screener questions!]
Should the number of screening questions change depending on who you’re trying to recruit?
On Respondent, different audiences attract different numbers of screener questions.
When you split the average number of screener questions according to Business Professional recruits and General Population recruits (as we do), the data yields some interesting results:
At first we were puzzled by the lower than average number of screener questions for Industry Professional studies, when compared to General Population studies. Our incoming hypothesis was that these numbers would be in reverse. Higher for Industry Professionals. Lower for General Population. However, when taking this result in the context of the Respondent platform, it makes sense.
Respondent provides a ton of verified work information on participants profiles including a verified email address and a clickable LinkedIn link. This additional information provided-as-standard on every respondent profile removes the need for researchers to ask information about people’s work history as part of the screener. This explains the need for fewer screening questions when recruiting Industry Professionals.
Should the number of screening questions change depending on your methodology?
When split out by methodology on the Respondent platform, the average number of screener questions asked by researchers looks like this:
A brief analysis of these results shows that agencies play a large part in these results. Researchers from Research and Design Agencies are:
- More likely to ask more screener questions than other researchers, and
- Are much more likely to conduct research using methodologies other than ‘one on one’.
This ‘agency effect’ explains at least part of the larger than average number of screener questions in ‘diary study’, ‘focus group’ and ‘in-home’ methodologies.
So there you have it.
10 screener questions is the average number of screener questions researchers ask, but if you aim for between 5 and 12 screener questions on your next project - you’ll be doing just fine.Did this answer your question?