Market Research: Why Talkative Respondent Detection Doesn’t Work

admin 0

We recently published a special report titled 25 Common Field Mistakes to Avoid When Doing Your Qualitative Market Research. In bullet point #12, we suggest that researchers eliminate open-ended screening questions that are intended to identify respondents who are extroverted and able to express themselves.

Here is the full text of point #12:

“If your screener contains open-ended questions meant to elicit expressive types of people, delete those questions. Questions like that don’t work and unnecessarily lengthen your screening process. Yes, you want to exclude respondents who can’t or won’t speak up, but you don’t need to one additional question to identify these people. Well-trained recruiters will weed them out within the first few minutes of screening. If you want more assurance that your respondents will be outgoing and outgoing, overdo your recruiting and include a pre-discussion phone interview. This would be done by the moderator, who would then select the appropriate respondents.

Our suggestions elicited many comments. Some readers agreed with us, others did not. One of the most interesting comments we received came from a qualitative fieldwork manager at a major full-service research firm. It went like this: “I disagree with one item on your list about excluding open-ended questions from evaluators. Good recruiters can easily identify articulate respondents, but tired or distracted recruiters can occasionally use autopilot.” … I think a little bit of quality in a filter is worth the time.”

Do you agree with this reader’s comment? You should?

What is an articulation question?

Articulation questions measure the respondent’s ability to communicate. Articulation questions also judge respondents’ responses. expected communicativeness in a focus group or interview.

Some synonyms for “communicative” include: outgoing, open, forthcoming, talkative, wild, talkative. So who decides what is communicative? The recruiter? The recruiting supervisor? The client who reads word for word in your daily reports? And how much communication is enough? How much is too much?

Even the most experienced recruiters cannot determine how outgoing, open, approachable, talkative, wild, or talkative a respondent will be at a future point in time. That is a decision that recruiters are not qualified to make. But they can be counted on to spot respondents who have…

  • language barriers
  • casual attitudes toward the recruiter, the recruiter’s questions, or the investigation
  • reservations about your ability to attend the investigation
  • none communication problems during the selection process

What to watch out for…

Respondents get tired or go on autopilot when selection interviews take too long (10 minutes or more).

Articulation questions do not belong at the end of your assessment. For some reason, the articulation assessment is almost always done at the end of the assessment interview. But why is a question that is supposed to be so important placed at the end of the assessment, when the chances of respondents being tired or distracted are the highest? What are recruiters learning about respondents at this point in the process that they don’t already know?

Articulation questions also don’t belong in front of your evaluator. Well-trained recruiters immediately engage respondents in a conversation about the details of the research. It is during this prelude to screening questions that recruiters address respondents’ questions and concerns and make an assessment of respondents’ communication skills.

Articulation problems are not magic bullets to ensure good participants in the focus groups. These questions simply ask recruiters to use their own biased judgment in deciding whether a respondent can communicate clearly.

Articulation questions lengthen your assessment. Remember this. The longer your filter, the higher your costs.

Respondents become anxious when you are asked questions outside the left field that are not related to the quiz questions. When asked, “What is a gazinkle?” or “How many different things can you do with a clip?” or “If you were a tree…?” could perplex even the most eloquent respondent. Crazy questions from confused recruiters and frustrated respondents. This line of questioning is moderator territory.

Of course, the dynamics of the group and the personalities of the respondents affect how open and receptive the respondents will be. For example, a person may get on the phone with the recruiter, but feel intimidated if an aggressive personality dominates the group. Or, a respondent may not feel as comfortable with the research topic as she thought and feel out of place, especially if the topic offered during recruitment was vague. How can recruiters know how respondents will perform in a variety of conditions? Handling reserved respondents is the moderator’s area of ​​expertise.

In fact, moderators are better qualified to know what can and should be expected from respondents in terms of communicativeness and articulation. So it makes sense that, as we suggested in point 12 of our special report, moderators should pre-interview respondents and select the right personalities for research.

So what about articulation questions that are helpful in snapping distracted or unaware recruiters out of their daze (as our reader suggested)? Assuming a tired and distracted recruiter missed all the red flags during screening, will the articulation question suddenly remind the recruiter that the respondent isn’t talkative? What should you do with recruiters on “autopilot”? Easy.

The researcher’s job is not to craft questions that keep recruiters alert and focused. Tired or distracted recruiters are not an asset to your research. They don’t help you get great respondents. And neither do articulation problems. Don’t use either one.

Leave a Reply

Your email address will not be published. Required fields are marked *