User Testing (Usability Testing)
User testing (also called usability testing) is a method in which you observe individual users as they try to complete tasks using a prototype or product. The users are asked to "think aloud" to explain what they are seeing, thinking, and doing while trying to complete the tasks.
The goal of user testing is to gather data to evaluate whether the prototype/product is easy to understand and use. You'll use this data to generate ideas for possible changes in the design of the prototype/product that might fix any problems observed during user testing.
User testing is helpful throughout the design and development process. Early in the design process, you can conduct user testing with low-fidelity interface sketches to explore different ideas. Later in the design process, you can test high-fidelity interactive prototypes to refine a specific concept. During and after development, you can conduct user testing with the current version of your product to verify and refine features. Ideally, your team should conduct user testing as often as possible during the design and development of your solution.
Once you have a particular prototype or product to evaluate, user testing requires four major steps:
Create Task Scenarios
Recruit Participants
Facilitate Testing Sessions
Summarize Findings
Create Task Scenarios
Identify each task that you want to test with your prototype/product during the session, and determine a logical order for the tasks.
Write a brief scenario (2-4 sentences) for each task that provides a context (plus any necessary details) for completing the task.
Print two copies of each task (one copy for the participant & one copy for your team). List each task on a separate slip of paper, so you can give the tasks one at a time to the participant.
You'll only be able to test a limited number of tasks during a particular session, so prioritize the most important tasks that should be tested. Depending on how each testing session proceeds, your team might need to skip certain tasks or change the order of the tasks.
The scenario should not provide specific steps or any clues for how to complete the task. However, the scenario should provide a context/reason for why the person is trying to complete the task, as well as any detailed information needed to complete the task.
For example, a task for an airline website might be to “book a flight.” The task scenario might be something like:
You're planning a family vacation to Colorado. You need tickets for 4 people to fly from Indianapolis to Denver. You want to leave on a Saturday in July and come back one week later. Use the website to purchase the least expensive tickets for this trip.
This example task scenario provides a context for the task (family vacation) and necessary details (number of people, departure airport, arrival airport, timeframe for departure and return, least expensive tickets, etc.) – without giving away the specific steps for how to complete the task.
The task scenario should avoid using the same key words that are present in the interface screens of the prototype/product because it acts as a direct clue to the participant of what to click or tap. For example, if an airline website has a button labeled as “Book a Flight,” the scenario should use a different (yet equivalent) wording for the task, such as: find flights, compare options, reserve a trip, buy tickets, etc.
Recruit Participants
Recruit 3-5 people (outside your team) to be participants that will test your prototype/product.
Ideally, the participants should be similar to your target users and also reflect the diversity (gender, age, etc.) of your target user population. For example, if your product is specifically designed for older users, you shouldn't test it with younger users. If your product is designed for both women and men, you shouldn't test it with only male users.
During recruiting, let the person know how much time the testing session will require:
If the person is volunteering their time, you should keep the session brief (less than 15 minutes). If possible, it's a nice gesture to give participants a small reward (e.g., food, bottle of water, etc.).
Testing sessions done by companies usually last between 30-60 minutes, so those participants are typically paid for their time.
If you're recruiting people in advance, be sure that the participant knows the exact date, time, and location for the testing session. Send the participant a reminder before his or her session.
Facilitate Testing Sessions
You’ll conduct an individual testing session for each participant. Be sure you have all your materials ready before the session starts (e.g., prototype or product to be tested, computer or other device for participant to use, printed task scenarios, paper and pencil to record notes, etc.).
At the start of the session, welcome the participant, and explain the overall testing process:
The participant will complete a series of tasks using the prototype/product. The participant will have to figure out the specific steps for each task without help from your team.
The participant should “think aloud” as he or she interacts with the prototype/product, so your team can understand what the person is seeing, thinking, and doing.
Clarify that the prototype/product is being tested – not the participant. Explain that the goal is to gather information to help improve the prototype/product.
Have the participant work on one task at a time. Give the participant a printed copy of the first task scenario, and read it out loud to the participant. As the participant tries to complete the task, your team should observe:
Your team should record notes describing any problems or issues that you see or hear. Don't provide any hints or feedback to the participant as they work on the task.
If necessary, remind the participant to “think aloud” as they work on the task. Because it’s not a natural habit to think out loud, some participants will forget to keep doing it. If the person has stopped talking, you could ask them “What are thinking right now?” to get them talking again.
If the person asks for help, just remind them that you can’t give them any hints. However, you can clarify the task scenario if it’s unclear. If necessary, remind the participant that the purpose of the session is to learn how someone would try to complete task when no one else is around to help. If necessary, you could redirect them by asking "What do you think the next step might be?"
You should end the task when one of the following has occurred:
The participant has successfully completed the task.
The participant believes the task is completed (even if it isn’t).
The participant is confused, frustrated, or unable to complete the task (and more time won’t help).
Regardless of whether or not the participant completed the task, you could say something like “Thank you. That was very helpful. Let’s move on to our next task.” Then give the participant the next task scenario, and repeat the previous steps.
Be sure to leave a few minutes at the end of the session for your team to ask any follow-up questions about specific issues that occurred during the testing. It’s better to ask these questions at the end, rather than interrupting the participant during the tasks. This is an opportunity for your team to probe deeper to understand why the participant did or said certain things as they interacted with the prototype/product.
In some cases, your team might have participants also complete a post-testing survey or interview to gather additional evaluation data that can be analyzed (quantitative and/or qualitative).
Be sure to thank the participant for their time and feedback.
Summarize Findings
Once you’ve tested the prototype/product with each of your 3-5 participants, your team should review and discuss the observations and notes from the sessions, in order to summarize your evaluation findings.
For each issue that occurred during the testing session:
Briefly describe the issue.
Estimate the severity of the issue.
Identify a possible design change that might solve the issue.
Estimate the feasibility of implementing the change.
Determine the priority of implementing the change.
Severity of Issues
Jakob Nielsen is a usability expert that recommends rating the severity of usability issues as Low, Medium, or High based on: (1) the issue's impact on the user experience, and (2) how many users are likely to experience the issue. This table shows Nielsen's severity rating matrix:
SEVERITY RATING | Few Users Likely to Experience Issue | Many Users Likely to Experience Issue |
Small Impact on User Experience | Low | Medium |
Large Impact on User Experience | Medium | High |
Design Changes
Steve Krug is a usability expert that recommends a process of tweaking a design to fix problems, rather than completely redesigning something. This diagram shows Krug's process:
Feasibility of Changes
Once you've discovered problems with your prototype/product and identified possible changes to fix the issues, you might realize that some changes will be easier or harder than others to implement.
There are many factors that can affect the feasibility of implementing a design change, but two main factors are: (1) how much time will it require, and (2) how much technical expertise will it require.
Depending on your project schedule, you probably have a limited amount of time to implement changes. Depending on your team's current technical abilities and available resources, there might be certain changes that will be more difficult to implement at the moment. This table shows a feasibility rating matrix for a design change based on these two factors:
FEASIBILITY RATING | More Technical to Implement | Less Technical to Implement |
More Time Required to Implement | Low | Medium |
Less Time Required to Implement | Medium | High |
Priority of Changes
You should prioritize the changes that will be implemented in your team's prototype/product since your team may not be able to address all the issues in the current design cycle. This table shows a priority rating matrix based on the severity and feasibility of each issue:
PRIORITY RATING | Low Feasibility | Medium Feasibility | High Feasibility |
Low Severity | Low | Low | Medium |
Medium Severity | Low | Medium | High |
High Severity | Medium | High | High |
As time and resources allow, your team should try to implement the recommended design changes for the highest priority issues first. If possible, test the revised prototype/product with a new set of participants to verify whether the issues have been resolved.
Last updated
