Usability Testing Guide For B2B & B2C Digital Products

In this Usability Testing Guide, I covered the following topics - 

1. What is Usability Testing?
2. Top statistics of Usability Testing
3. Top 8 benefits of website usability testing
4. What usability testing is not
5. Criteria for the usability of your website
6. Usability Testing methods at a glance
7. Top 6 Usability Testing tools
8. Usability testing pro tips
9. Hire Usability Tester

10 min read

How user-friendly is your website? The best way to find out is with a usability test. External help, a large budget, or extensive analysis are not required for this. Instead, you can simply check the usability of your website yourself. We reveal what is important.

What is Usability Testing? 

Usability testing is all about getting real people to interact with a website, app, or other product you've built and observed their behavior and reactions to it. Whether you start small by watching session recordings or go all out and rent a lab with eye-tracking equipment, usability testing is a necessary step to make sure you build an effective, efficient, and enjoyable experience for your users.

Usability is not only about ease of use but also about bringing something meaningful, having an objective in common with the user.

-- Marcus Österberg, Web Strategy for Everyone

How to conduct Usability Testing?

In a usability-testing session, a researcher (called a “facilitator” or a “moderator”) asks a participant to perform tasks, usually using one or more specific user interfaces. While the participant completes each task, the researcher observes the participant’s behavior and listens for feedback.

The phrase “usability testing” is often used interchangeably with “user testing.”

Why Usability Test?

The goals of usability testing vary by study, but they usually include:

• Identifying problems in the design of the product or service
• Uncovering opportunities to improve
• Learning about the target user’s behavior and preferences

Usability Testing Process

To make usability testing work best, you should:

1. Plan –

a. Define what you want to test. Ask yourself questions about your design/product. What aspect/s of it do you want to test? You can make a hypothesis from each answer. With a clear hypothesis, you’ll have the exact aspect you want to test.

b. Decide how to conduct your test – e.g., remotely. Define the scope of what to test (e.g., navigation) and stick to it throughout the test. When you test aspects individually, you’ll eventually build a broader view of how well your design works overall.

2. Set user tasks –

a. Prioritize the most important tasks to meet objectives (e.g., complete checkout), no more than 5 per participant. Allow a 60-minute timeframe.

b. Clearly define tasks with realistic goals.

c. Create scenarios where users can try to use the design naturally. That means you let them get to grips with it on their own rather than direct them with instructions.

3. Recruit testers – Know who your users are as a target group. Use screening questionnaires (e.g., Google Forms) to find suitable candidates. You can advertise and offer incentives. You can also find contacts through community groups, etc. If you test with only 5 users, you can still reveal 85% of core issues.

4. Facilitate/Moderate testing –Set up testing in a suitable environment. Observe and interview users. Notice issues. See if users fail to see things, go in the wrong direction or misinterpret rules. When you record usability sessions, you can more easily count the number of times users become confused. Ask users to think aloud and tell you how they feel as they go through the test. From this, you can check whether your designer’s mental model is accurate: Does what you think users can do with your design match what these test users show?

If you choose remote testing, you can moderate via Google Hangouts, etc., or use unmoderated testing. You can use this software to carry out remote moderated and unmoderated testing and have the benefit of tools such as heatmaps.

Top statistics of Usability testing

• 5 users will find 85% of usability problems when testing continuously. In a Poisson Distribution with a 31% binomial probability, 5 users in a test group are able to find 85% of the problems. However, as you add more than 5 users to a test group, you start exhibiting drastic diminishing returns, which means the more people you add, the less you learn.
• 50% of a developer’s time is spent trying to fix issues that could have been avoided. In a study from the University of Cambridge, researchers were able to come up with a similar conclusion as those surveyed responded that 49.9% of their programming time was spent debugging. The problem is that this results in project delays and added costs. It is estimated that over $625 billion is spent on software production globally in terms of wages and overhead, with $312 billion spent on debugging alone.
• Problems in development are 10 times more expensive to fix than during design. Any error that you find early in the process, such as during the design phase, is 10 times less expensive to correct than fixing it during the design phase. Costs pile up even higher when trying to fix the problem after the product has been released.
• Poor user experience with ads has contributed to the stigma as 420 million people around the world now use ad blockers. Does your product depend on ads for revenue? Then ad blockers might be your worst enemy. An estimated 420 million people around the world now use ad blockers to improve their experience while using an app or viewing a web page. This results in revenue losses of up to $35 billion by 2020 for publishers. 
• Even when they like the business, 50% of users will not use a website if it isn’t mobile-friendly. In 2019, the number of mobile phone users around the world is expected to reach 4.68 billion according to Statista. This means over 60% of the world’s population will be dependent on their mobile phones. This also means that a great mobile experience is absolutely necessary when you want to do business online.
• “Ease of use” is cited as the most important quality for mobile apps by 97% of users. There is no doubt that users place a high priority on how easy it is to use an app over other features. This is according to The State of Mobile Enterprise Collaboration published by Harmon. In the survey, 97% of business respondents place ease of use as the highest priority over mobile application security (89%), comprehensive features (72%), and training (42%). In a separate study by Accenture, users placed ease of use as the highest priority (33%) when buying a smartphone device. 
• 79% of users that fail to find what they were looking for will look for another website. Speaking of ease of use, 79% of users admit that they leave and search for another website if they are unable to access or find what they were looking for. 
• 86% of users want to find more information about a site’s products and services after landing on a homepage. Consumers want to learn everything about a brand and its products before they make a purchase. That is why 86% of users look for more information when browsing a website, 62% of users look for contact information, and 52% are more interested in the “About us” section. 
• The lack of a message can cause 46% of site visitors to leave. A 2015 research has uncovered that 46% of users refuse to stay on a website that lacks an effective message, i.e. they can’t tell what the company is or what it does. In the same study, 44% of users reported they would leave when there is a severe lack of information, and 37% left due to poor design or navigation.
• Content sliders don’t work – only 1% of users click on them. Content sliders, or image carousels, are primarily used for content navigation and exploration. 

Usability is about people and how they understand and use things, not about technology.

-- Steve Krug, Author

Your website can benefit from usability testing no matter where it is in the development process, from prototyping all the way to the finished product. You can also continue to test the user experience as you iterate and improve your product over time.

Top 8 Benefits of website usability testing

Your website can benefit from usability testing no matter where it is in the development process, from prototyping all the way to the finished product. You can also continue to test the user experience as you iterate and improve your product over time.

Employing tests with real users helps you:

1. Validate your prototype. Bring in users early on in the development process, and test their reactions before locking down a final product. Do they understand what you're trying to do? Do they see the purpose of the product? Testing on a prototype first can validate your concept and help you make plans for future functionality before you spend a lot of money to build out a complete website.
2. Confirm your product meets expectations. Once your product is completed, test usability again to make sure everything works the way it was intended. How's the ease of use? Is something still missing in the interface?
3. Identify issues with complex flows. If there are functions on your site that need users to follow multiple steps (for example an e-commerce checkout process), run usability testing to make sure these processes are as straightforward and intuitive as possible.
4. Complement and illuminate other data points. Usability testing can often provide the why behind data points accumulated from other methods: a heat map might show that people aren't paying attention to an important part of the landing page, but observing users in action can reveal why they're ignoring it.
5. Catch minor errors. In addition to large-scale usability issues, usability testing can help identify smaller errors. A new set of eyes is more likely to pick up on broken links, site errors, and grammatical issues that have been inadvertently glossed over. Individually, these incorrect details may not matter a lot, but, collectively, they contribute to the perception of a site's professionalism and trustworthiness. 
6. Develop empathy. It's not unusual for the people working on a project to develop tunnel vision around their product and forget they have access to knowledge that their typical website visitor may not have. Usability testing is a good way to develop some empathy for the real people who are using and will be using your site and look at things from their perspective.
7. Get buy-in for change. It's one thing to know about a website issue; it's another to see users actually struggle with it. When it's evident that something is being misunderstood by users, it's natural to want to make it right. Watching short clips of key usability testing findings can be a very persuasive way to lobby for change within your organization.
8. Ultimately provide a better user experience. Great customer experience is essential for a successful product. Usability testing can help you identify issues that wouldn't be uncovered otherwise and create the most user-friendly product possible.

Pay attention to what users do, not what they say.

-- Jacob Nelson, Co-founder of NNg

What usability testing is not

There are several UX tools that help improve the customer experience, but don't really qualify as 'user testing tools because they don't explicitly replicate the experience of real users testing a website for functionality: 

• A/B testing: A/B testing is a way to experiment with multiple versions of a webpage to see which is most effective. Unlike usability testing, which observes and investigates user behavior, A/B testing can help validate whether a certain approach is working or not, but can't tell you why.
• Focus groups: when conducting a focus group, researchers gather a group of people together to discuss a specific topic. Usually, the goal is to learn people's opinions about a product or service, not to test how they use it.
• Surveys: use surveys to gauge user experience. Because they do not allow you to actually observe visitors on the site in action, surveys are not considered usability testing—though they may be used in conjunction with it.
• Heat maps: Heat mapping software offers a visual representation of how users move around the page by showing the hottest (most popular) and coolest (least popular) parts of it. The maps allow developers to see how people in aggregate move around a website, but they are still technically not usability testing.
• User acceptance testing: this is often the last phase of the software-testing process, where users go through a calibrated set of steps to ensure the software works correctly. This is a technical test of QA (quality assurance), not a way to evaluate if the product is user-friendly and efficient. 
• In-house proper use testing: people in your company probably test software all the time, but this is not usability testing. Employees are inherently biased, making them unable to give the kind of honest results that real users can.

Criteria for the usability of your website

Translated, usability means suitability for use. It describes the extent to which a user can use a product or service in a specific application context to satisfactorily achieve their goals. Put simply, a usability test can help you determine how well visitors get along with your website.

The term usability has only been used since the 1980s. How people get along with technology was of course already a topic beforehand. At least since the invention of huge machines like the steam locomotive and the number of terrible accidents caused by operating errors, it was clear: usability is an important criterion when it comes to making technology accessible to people.

In contrast to the steam engine, there is of course no danger from your website. If the user-friendliness is right, you can increase the length of time your visitors stay, reduce the bounce rate or even generate leads.

The following components of your website play a role in this:

Content

The Internet has many advantages - and one problem: too much input. Decades ago, an information overload of 98 percent was calculated for Germany. That means 98 percent of the information that we encounter every day, we don't even notice.

Of course, this also influences the reading behavior of your visitors. So you have to give them a reason to take a closer look at your content of all things.

It starts with the address of your website. If it is unattractive or overwhelming, it will hardly be clicked. While the latter probably overwhelms well-meaning users, the former presents all the essentials at a glance.

Overall, the following questions are important with regard to the usability of your content:

• Is your content relevant and not unnecessarily long?
• Are they well-structured and understandable?
• Do they offer your visitors added value?
• Do they cover the search intent of your visitors?

Usability is like love. You have to care, you have to listen, and you have to be willing to change. You’ll make mistakes along the way, but that’s where growth and forgiveness come in.

--Jefferey Zeldman, Principal Designer at Automattic

Design

Anything that pleases is allowed? It's not that easy. A clear/simple website design helps your visitors to understand content immediately and without any problems. You should avoid constantly popping up windows, flashing buttons, or umpteen different fonts and colors.

Instead, the following questions are important:

• Does the overall design of your website provide a pleasant user experience?
• Does the design help your visitors find their way around the site?
• Are there visual hierarchies that help your visitors better understand the content?
• Do you use images, videos, or similar things that offer added value to your visitors?

Navigation

One of the most important factors in usability is navigation. Your visitors can't find their way around if it's complicated or overloaded - and they leave your website again very quickly. As with design, the goal of your navigation should be to help visitors find information.

The following questions are important here:    

• Is your navigation generally comprehensible?
• Is it logically structured and divided into hierarchical levels?
• Is the number of menu items manageable? Between five and seven are easily perceptible.

Web conventions

You most likely use web conventions anyway. This is about standards in website design that every user knows and somehow expects. You should not break with these conventions in order not to confuse your visitors.

Common web conventions include these:

• Your logo is at the top of the website. Users can click it to go back to the home page.
• The navigation is either at the top of the page or on the left side.
• Standard links to the data protection declaration, terms, and conditions or imprint are placed in the footer of the website.
• Hyperlinks are color coded.

Don’t find customers for your products, find products for your customers.

-- Seth Godin, Blogger/Entrepreneur & Author

Creativity

Another quality feature is the creativity of your website. It should offer visitors something they won't get anywhere else—something that will intrigue them and make them stay on your site longer.

As part of usability, you should ask yourself the following questions:

• Is your website remembered by your visitors?
• Does it offer your visitors something new that they haven't seen before?
• Are there applications or similar that help your visitors - and differentiate you from the competition?

Usability Testing Methods at a Glance

Whether – and how many – of the questions presented can be answered positively, you should not simply judge for yourself. A usability test provides much more accurate results. There are very different approaches that can differ significantly in terms of effort and process.

In general, there are these variants:

• Expert tests: If you don't want to lend a hand yourself, turn to experts. They have different ways of examining a website. One of them is to assess the quality of the website using heuristic methods: The experts use existing guidelines, check the website against these specifications, and note any anomalies.
• Qualitative test methods: With qualitative test methods, the focus is on the individual test results. What did this user do to get the information they wanted? What difficulties did this user encounter and how did it detract from her user experience? Small test quantities are enough to find out the central problems of a website.
• Quantitative test methods: Quantitative test methods require higher numbers of participants because they are intended to determine measurable numerical values. They are well suited to revealing general difficulties. For example, if none of the test subjects were able to solve a task, this is a clear sign that improvements should be made in this area of the website.
• Online tools: Finally, there are online tools that can be used to identify a website's weak points. Most of them are quick to set up, easy to use, and often even free in their basic version. There are now numerous online tools that specialize in a wide variety of key figures. More on that is below.
• Remote Test: Remote testing is any testing that happens when the participant and the researcher are in separate locations. In a remote test, the participants complete the tasks in their natural environment using their own devices. The sessions are facilitated by online tools and can be moderated or unmoderated.

Top 6 Usability Testing tools

If you want to find out quickly and without much effort what the usability of your website is like, the following approaches are recommended:

UserTesting is one of the most popular platforms among usability testing tools.

Using it, you can quickly get feedback from real users in video or note format. I prefer to work with videos or participate in live conversations, as it allows me to observe users’ facial expressions, analyze intonation and see their live reactions.

One useful feature is Highlight Reel. It allows you to highlight key points that are important to bring to the team’s or client’s attention.

By the way, if you use Justinmind for prototyping, you’ll be interested to know that UserTesting is integrated with it.

HotJar is one of the key usability testing tools. Over 900,000 organizations currently use Hotjar for usability analysis and in 2021 the tool was recognized as one of the leading services in EMEA.

I would highlight the key features you get when using the tool:

• Heatmaps: maps clicks, scrolling, mouse movement
• Feedback collection
• View records during A/B testing
• Tracking and recording of user behavior
• Analysis of conversion funnel
• Form Analytics
• Create various questionnaires and surveys
• Click comparison

The pluses for me personally are the ease of setup, which takes only a few minutes, and the visualization of the data, which makes it easier to analyze user behavior.

UserZoom — perhaps requires no introduction. But I thought I’d add it to complete the picture. It’s a huge platform, suitable for large companies, as it has around 120 million potential test participants.

Once you sign up and pay for the service, you get access to the complete package for online user research — from recording sessions on multiple devices to saving your driving route through the site/app, not to mention heatmaps. Plus in UserZoom you can run multiple studies at the same time.

I usually set up the study package so that only the add-ons I need for the project are there. This allows me to be more productive and not be distracted by redundant data.

People should never feel like a failure when using technology. Like the customer, the user is always right. If software crashes, it is the software designer's fault. if someone can't find something on a website, it is the web designer's fault_ The big difference between good and bad designers is how they handle people struggling with their design. Technology serves humans. Humans do not serve technology.

-- Joshua Porter, co-founder of Rocket Insights

Automated Examination: Online Tools

Online tools provide good indications of where on your website the problem is. They cannot replace the concrete experiences of real users; they provide good starting points where there is a need for action. Big plus point: You can start testing immediately and without preparation.

One tool you should take a closer look at is Google's Lighthouse. Originally, the service was developed for testing progressive web apps; meanwhile, however, conventional websites can also be put through their paces with it.

You can measure numerous different factors and also receive specific recommendations for action to optimize your website. In order to receive evaluations that are as realistic as possible, you also have the option of filtering and adjusting the test conditions.

Quantitative method: Onsite survey

A quantitative method with test persons is the on-site survey. Here you can generate feedback from many visitors to your website within a short time.

For this purpose, a questionnaire is integrated directly into your website and visitors are invited to participate in the survey via a popup or a layer. You can decide for yourself whether you want to take part in the survey.

With an onsite survey, you can find out, for example,

• how well visitors generally get along on your website,
• if they have any suggestions for improving your website,
• which functions they miss on your website.

You can use the onsite survey not only to gain insights into your website but also to get to know your target group better. Ultimately, your questions end up with the people who visit your website anyway. In addition, the on-site survey contributes to regular quality assurance because you can use your questionnaire again and again without any effort.

Qualitative method: observation and questioning

A qualitative method that is easy to implement is personal observation and questioning. Test persons are assigned certain tasks, which they are then supposed to solve under observation.

The good thing about it is: Large test quantities are not necessary! According to a study by usability expert Jakob Nielsen, just five test subjects can uncover 80 percent of all usability problems. So you only need five test results to increase the user-friendliness of your website enormously.

For a meaningful result, you should proceed as follows:

• Find people who would like to help you. But don't make your life unnecessarily difficult by endlessly searching for the perfect subjects. It's about testing a regular website, so almost anyone can do it. Friends or relatives are good candidates.
• Set your test subjects' tasks and observe how they go about solving them. So that they can better adapt to the situation, it is helpful if you invent a small scenario. This can look like this, for example: “For the sake of the environment, you would like to switch from the car to the bicycle. Find a bike for your commute that suits you and is in the mid-range price segment.”
• Important: Your subjects should be able to find their way around on their own. Under no circumstances should you give them specific instructions, because this will falsify the results.
• You can find out even more details in the subsequent survey: How did your candidates find the level of difficulty of the tasks set? How did you come up with the respective solutions? If there were difficulties – why was that? The knowledge gained will help you to improve your web presence.

Usability testing pro tips

• Recruit realistic participants. People who would actually perform these tasks in real life
• Use 5-8 participants for qualitative usability testing
• Avoid influencing participants during testing. The easiest way to do this is to stay quiet during the study
• Ask open-ended, neutral questions when you speak to the participant. For example, say “What do you think about this?” instead of “Do you like this?”

Testing with more than 5 users results in diminishing returns

Sure you can test with more people, and there is a chance that you won’t uncover any issues until your 6th or 7th user. But it is very unlikely if you’ve designed your tests to cover the core areas of the service. What is more likely is that by your 6th user you’ll start to see the same patterns of behavior repeat over and over, and your value from the tests start to diminish.

You may have seen the great diagram put forward by Jakob Nielsen to show the diminishing returns of testing with more than five users.

We agree with this entirely, and what many people forget about the accompanying article (Why You Only Need to Test with 5 Users) is that he advocates testing with five people, fixing the issues, and then testing again. This is very much the way we like to work. We encourage many clients, who come to us with plans to recruit 15/20 people, to reduce their scope and break the project into chunks that allow for design changes to be made before new users are brought in.

You don’t need high numbers of users to identify usability problems. Don’t confuse usability research with other types of market research. If you want to gauge opinions about competing brands, then five people aren’t anywhere near enough. But if you want to identify barriers to purchasing on your website five users are usually enough.

Conclusion

If you want to find out how user-friendly your website is, you can't avoid a usability test. You can hardly test your website yourself - after all, you already know how it works. Instead, online tools or test subjects who have a fresh look at the site will help you uncover problems that you wouldn't spot on your own.

Of course, you cannot eliminate all difficulties that arise overnight. Improving user-friendliness is a constant development process that can significantly increase the success of your website.

Credits

• Digital.Gov
• W3.org
• UXdesignCC
• NN/group
• Dribbble