The Power of Field Testing

Powell and Miller talked about qualitative field testing of materials. This type of field testing focuses on quality and has few participants. The data collected is subjective, personal, and in-depth, they said. They offered tips for conducting qualitative field testing. When choosing a test site, we should think about the ideal type of site and site features. “The test site can be just about anywhere. But make sure it creates an environment of free expression so participants will tell you their opinions. Think about audience, budget, and time frame,” said Powell.

The presenters offered tips for recruiting participants, like asking the site for help, provide incentives, screen people for desired demographics, recruit extra participants, in case of no-shows, get the first names of participants and send reminders. They also offered tips for making a detailed schedule, such as limiting each session to about one hour, and allow extra time for notes, bathroom breaks, snacks, and late arrivals. If you have more than one researcher, hold concurrent sessions to save time. “Have the researchers observe each other so they can align their styles,” said Powell.

The presenters offered guidelines for two types of qualitative field testing: cognitive interviews and focus groups.  Cognitive interviews are conducted one-on-one. The interviewer acts as an observer. Her job is to ask follow-up questions and note nonverbal reactions, the presenters said. “These can help you understand the participants’ perspective and give you thoughtful insights,” Miller said. Focus groups are conducted in a group setting — usually eight to 12 participants. The moderator leads a discussion, rather than just observing. His job is to encourage group interaction, ask follow-up questions, note nonverbal reactions, and make sure no one dominates.

The presenters discussed how to work with interpreters. They talked about how to write field test questions and record answers. They also discussed how to analyze and summarize data and write recommendations. The final report should include participant quotations to illustrate points, they said. “This is often what people look at first or think is most important,” said Miller.