How to conduct user testing yourself (for free!)

This article was written for BoagWorld

Usability testing doesn’t need to be time consuming or expensive. You can do it yourself, for free. In an ideal world, we would test our websites (and apps) with actual users on a regular and ongoing basis. In the real world, user testing can feel like such a daunting and time-consuming task that we don’t bother testing with users at all.

If you’re lucky, you (or whoever built your website) may have done some user testing when your website was created. Testing as you design and build a website helps to keep your website redesign on the right track. Without user testing, we risk breaking the mental model of existing users, and creating something that confuses and frustrates everyone.

Why is User Testing so important?

Every website has usability problems which make users frustrated.
Frustrated users leave websites.

The issues on your website may not be obvious to you, usually because you are probably very familiar with your own site. You know how it works. Most of your visitors don’t visit your site on a regular basis, especially potential customers, so they don’t know how it’s supposed to work – to them, it’s all new.

User testing as part of a website redesign process is clearly commendable, but what happens next? All too often, the website project is marked as complete and no-one gives it another thought until the next big redesign.

Here lies a missed opportunity.

Following a process for continuous improvement could mean that you never need to redesign your website again. By implementing ongoing testing and iteration, you can be sure you’re delivering an experience that aids and delights your users, which will, in turn, ensure the needs of your organisation are met. It’s a win-win.

If your budget allows it, you can hire a usability expert to help keep your new website ship-shape. Aside from knowing their craft, a fresh pair of eyes will always be able to see things from a different and unbiased perspective.

If external help is not an option, this article is for you.

In this article, we will walk through a process for DIY user testing. If you want to know how you can test your website (or app) yourself, quickly and with minimal expense, read on.

DIY User Testing

Watching people use your website or digital products will always be valuable. Given the speedy and simplified nature of DIY user testing, it probably won’t result in much qualitative data (what they do – use Google Analytics for that), but you will get valuable insights (why they do). These insights will help you create a better experience for your users, improving the conversion rate on your site, which in turn will help your organisation meet its objectives. An improved conversion rate means any future marketing and advertising spend will result in more effective campaigns with a better ROI.

What do you need to conduct DIY user testing?

To conduct simple, DIY user testing on your website, you will need to:

have a computer with a microphone/speaker
get in a room with a participant,
give him or her some jobs/tasks to do
watch and take notes
encourage your participants to think out loud, (so you can start to understand what’s going through their mind as they use your website/app).

You could even next-level this by gathering your stakeholders in another room where they can watch the session via screen share. Once the test is complete, you can discuss any problems that were found as a team, and brainstorm how you might approach fixing them.

That’s it. No smoke, or mirrors (or two-way mirrors).

This would be a very short and uninteresting article if we stopped here, so read on to find out how you can implement this technique to increase conversion on your website.

How often should you test and how many people do you need?

According to usability expert Steve Krug (who’s book, Rocket Surgery Made Easy prompted me to write this post), effective user testing can be conducted in one morning each month. That’s four hours a month, which should be sufficient time to carry out around 3 user tests.

Keeping it simple but regular is the kicker. It also means you have around 4 weeks to implement fixes to any problems you find, and time to find your next 3 participants.

So, you may be thinking:

If you can do DIY user testing with 3 people once a month and get results, why would you ever do it ‘properly’?
How can you make decisions based on 3 people?

Both very valid questions, so let’s clear that up.

With DIY user testing, we’re not trying to prove anything. If you’re on a mission to demonstrate that your new booking form will increase appointments by 30%, then you’re obviously not going to be able to prove that by running 3 user tests. What you will discover with DIY testing, is any major usability issues that need fixing, which is a really good way to get the low hanging fruit.

What to test in a DIY user test.

It’s important to remember that you don’t have to have a fully working website to start testing. Testing early and often is the ideal approach. With that in mind, here are a few things you could test in a 15-minute user test, whatever stage your website redesign is at.

Tasks a user needs to complete on your own website.
Tasks a user needs to complete on a competitor website. (Just because it’s new and shiny and you like it, doesn’t mean it doesn’t have usability or conversion rate issues).
Sketches (What do you think of this…? What do you think it’s supposed to be…?)
Wireframes. (How would you find… ? What would you expect to see when you click on…? )

Who to recruit for DIY user tests.

When we think about the ideal user testing participant, we might think we need real users, by which we mean those who already use our site or at the very least fall within the scope of our target market/user persona.

This makes sense to a point. Why test people who aren’t going to use your site? Won’t people who share your user persona have the same issues as your perfect customer? And what about domain knowledge? Surely it’s best if your testers have some understanding of the problem your website it trying to solve?

Well, yes, but don’t get too caught up in this. Remember, your user base may be more varied than you think. Not everyone will know what we assume them to know. (That’s how we ended up here in the first place).

Real users are a great idea, but just as any testing is better than no testing, any user is better than no user.

Preparing for the tests

Practical preparation

From a practical perspective, you’ll need a room, a computer connected to the internet and ideally screen recording/screen sharing software. The former will help jog your memory after the test, the latter will ensure any stakeholders can watch from another room.

Set a task and write a scenario

Think about the most important things people will want/need to do on your website and write a list. This could include,

research a purchase,
buy a product,
sign up for an event,
purchase a ticket,
book an appointment,
find out how much a service costs etc.

Next, decide which ones to test. Which are most important? Which have the potential to break your website or stop your user from progressing? Which tasks do you already suspect are problematic and confusing?

Once you have decided on the tasks, the next step is to create scenarios around them. The scenario will be given to the participant and should include a character, a motivation, a job to do (task) and maybe some additional details.

A short user test might look like this:

Task: Make an appointment to see a skin therapist.

Scenario: You’re unhappy with the appearance of your skin lately and have decided to have a facial. Book an appointment via the online booking system.

Tip: Be careful not to give too much away via the terminology you choose. Be clear and avoid words which are internal to your organisation, product or service.

In the session, you should allow around 30 minutes for the user to complete all the tasks, so keep that in mind when you decide which ideas you will be testing.

Create a welcome script

Having pre-written scripts will ensure you don’t miss anything. It will also help to keep the tests consistent amongst participants.

Explain who you are, what will happen during the session and how long it will take.
Emphasise that you’re testing the site, not the participant, so there are no wrong answers.
Ask the user to think out loud, say what they’re looking at, explain what they’re trying to do and what they are thinking.
Explain you’re trying to improve the site, so honesty is best – they can’t hurt your feelings!
Ask for permission to record them and stress that the recordings won’t be used for any other purpose. (Perhaps have something they can sign to agree to this in writing).
Let them know if there are additional stakeholders observing from another room.
Ask if they have any questions.

Conducting a DIY user testing session

Here is an example of what a 1-hour user testing session might look like.

Warm welcome (5 mins)

Once settled in your testing room, read your prepared welcome script to your participant. Ask if they have any questions and answer these before moving on.

Pre-test questions (2 mins)

Ask your user a couple of questions about themselves to help them settle in and get them talking. What do they do for a living? How much time do they spend surfing the web each week? Do you have any favourite websites?

Conduct a home page tour (3 mins)

Ask your participant to take a look at the home page and think out loud as they look around. Ask a few questions to help them give feedback. For example, What strikes you about this page? What can you do here? Who is it for?

Complete the tasks (35 mins)

Explain you’re going to give them a series of tasks. Read them out loud, one at a time. It’s good to provide a print out too so they can jog their memory if needs be. Try not to interrupt unless it’s to help keep them focussed on the task you have set.

Get some answers (5 mins)

During the tasks, you’ll probably have come up with questions about what they were doing and why. Now is the time to ask them, as opposed to interrupting them during the task. You can ask why they did/didn’t notice certain things, or why they made a particular choice. You can also use this opportunity to check in with stakeholders who may be watching from another room, to see if they have any questions for the participant.

Wrap it up (5 mins)

Ask if your participant has any questions, thank them for their time.

Tip: Keep an eye on your users demeanour. If it looks like they are becoming stressed or uncomfortable, you can move onto the next task. Be sure they don’t think they have failed, just point out that there are other tasks and you should move onto the next to keep the session on track.

So there you have it, a DIY user testing session in under an hour. Take a short break and repeat this process with your other participants.

Debriefing after a user testing session

A short debriefing session soon after the testing is complete is ideal. The goal is to leave the session with a list of the main usability issues that became apparent during your testing, and a priority list of those that you will fix before next month’s testing.

Getting all stakeholders on the same page about what should take priority is important to ensure the work gets carried out.

You can kick off by asking everyone to review their list of usability issues noted whilst watching the session, selecting 3 top priority items from their list.

Create a list of these priorities, skipping any duplicates. Once the list is complete, select the top ten. You can carry out a dot voting exercise if needs be.

With your top ten usability issues selected, discuss how these might be fixed. Try to avoid grand schemes and major changes, focussing instead on the simplest way you can eradicate each issue.

User testing in 4 hours a month.

So, 3 users @ 1 hour per session, plus a debriefing session with your stakeholders is just 4 hours. Not forgetting that someone will need to find participants and make arrangements with them, setting up the room and conference facilities on the day. At a worst-case scenario, you’re looking at 1 day a month, to help your organisation move towards a better experience for your users, making it easier for them to find what they need, carry out tasks and do business with your organisation.

If finding users is too hard and/or you don’t have the facilities to conduct user testing at your workplace, then there are still other options. Online services such as UserTesting.com enable you set tasks just the same and receive screen recordings from users trying to complete the tasks you set. You obviously miss the opportunity to observe your users body language and facial expressions as they go about the tasks, but it’s an easier alternative that can still help you to gain a better understanding of any major usability issues you may have missed.

The question should not be whether you can afford to do user testing on your websites, it should be whether you can afford not to.