Do you recognize this?
Moderator: What’s your impression of this page?
Test user: Yeah, I like it.
Moderator: What do you like about it?
Test user: Well, you know. The colours are nice.
User testing is all about the quality of your participant’s output. You study how people use a website or device and try to figure out what’s going on in their heads. This is especially true when testing ideas and concepts: sketches of sites or applications to be. The material you test is not ready, but you’re dying for some feedback: Do they understand the concept? In this stage you often have to rely on participants’ spoken expressions, and this can be a daring endeavor.
We are really testing the website, not you
The ideal test user is equipped with…
- … a critical view of the world;
- … no hesitation to say what he really thinks;
- … a fine sense to put ideas and thoughts into words, and;
- … a lack of care about what the moderator thinks about his ideas.
Unfortunately – or maybe I should say: happily – test users are just common people too. They are susceptible to stress, social pressure, fear of failure, persuasion, insecurity, et cetera. And all of that probably even a little more during a user test session.
Sitting and waiting for the ideal test user to come along, won’t solve the problem. So the challenge is to surpass the human characteristics of test users and optimize the amount and the authenticity of a test user’s output. In other words: How to reveal the actual experience of your test users?
Before we dive into the tactics to have test users say things they didn’t even know themselves, let’s first discuss why it’s worth all the effort.
Why we should try to squeeze everything out of our test users
Each person participating as a test user is equipped with a package of experiences, values and preferences. After all, each test user is screened on particular relevant knowledge and motivations before participating in a test. However, those references, motivations and ideas are all very personal as well. Imagine yourself in a test session: would you tell a total stranger that you do not have the slightest idea of the meaning of ‘FAQ’? Or that you wish you’d know all about car insurance, but you just do not understand the differences between the 15 variations presented?
In short: relevant test-user output is crucial, but not self-evidently shared with the world. To the researcher, limited input from the end user, means: limited directions to improve the design. And that’s not going to make things better.
Ok. So how to get the most out of your test user? Based on extensive experience with user testing here at our usability lab, I’ll present three techniques that can be applied during a user test session.
Relating apples to airplanes
Generally, it’s all about giving the test user a positive feeling about himself. Provide him with the idea that his personal suggestions and feedback is particularly useful.
Do this by giving him a very simple exercise task in advance of the actual test. Be sure that this task is of a kind that cannot be performed in either a right or a wrong way.
For example: present repeatedly two pictures of unrelated objects simultaneously on the participant’s screen. Assignment for the participant: “Tell me the word that comes to mind when viewing these two pictures.” Every pair of pictures is presented shortly (about 10 seconds), to prevent the participant to think about his own thoughts.
Every output the participant gives regarding this task is the result of his own personal thoughts and beliefs: reward this.
Such an exercise will in most cases activate the participant’s proactive state of mind. Additionally, the test did not even start and he already experienced how it feels to share personal ideas.
Talk to a piece of paper
Do not encourage the participant to start a conversation with you while being a moderator. In stead, provide the participant with a sufficient load of paper and pencils and instruct him to, whatever comes to mind during the session, write it down. This will create a sense of freedom and an absence of responsibility and social pressure in the experience of the participant. As a moderator you can simply just leave the room after explanation of the assignment and have the participant do his thing all by himself.
“It’s on the tip of my tongue…”
A participant might decide consciously not to tell a certain idea or experience. On the other hand, the inspirational thoughts or relevant words might simply not pop into his mind during the course of a test session. Assist the participant, by providing cues and triggers. A couple of techniques:
- An extensive set adjectives (e.g. printed on cards) enrich the positive as well as the negative vocabulary of the test user. Give the test user 2 minutes to select 5 cards out of a set of 100+ cards and let him explain why he decided to pick out that particular cards. Read about how Microsoft used this technique “to measure desirability” (word document).
- Printed screenshots of the tested website invite the participant to start sketching or to write down remarks at specific website elements. Provide sufficient pencils and post-its and just leave the rest up to the magic between the prints and the participant. In this way it is not necessary for the test user to name all the individual page elements in order to give particular feedback.
Activation, not manipulation
Revealing whether a test person is able to work with a website or application still can best be done by viewing and interpreting the interaction. But, since user testing should be happening from the first scratch, it is worth the effort to knead the test user a bit into expressing his ideas while looking at this first scratch. As presented above, it just requires some simple tools and techniques to get some extra user feedback.
The tactics presented here are purely about kneading the participant to improve the quality of his output during a test, especially in the case of a concept test. Of course, there is a thin line between kneading and manipulating: a repeated chapter in qualitative interviewing techniques. If you have tricks to tackle that everlasting thin line, share it.