Usability testing is part of developing user-centric design. Conducting a usability test measures the task success, user satisfaction, and errors of a product. The data gathered from a usability test could answer questions such as:

  • Will the users recommend the product?
  • Do the users feel good about the product or themselves after using it?
  • What are the most significant usability problems with this product?

Challenge

The MRU Library has made a number of changes to technology and services over the past couple of years, specifically the website (library.mtroyal.ca). Continued financial pressures at MRU demand that resources and services be prioritized.

Solution

Data on effective student use of the MRU library resources could provide input for the strategic allocation of resources. Two approaches were taken to collect usability metrics from the MRU library website: task-based testing and group test activities.

2.1 Task-based Testing

26 participants were recruited in a moderated session. The participants were briefed, asked for consent, and compensated. The moderator provided tasks for the participants to complete using the MRU library website. Among the metrics measured by the tests were task success, confidence in task completion, performance, and system usability scale (SUS) score.

2.2 Group Test Activities

Groups of students identified areas for investigation and initiated their own testing scenarios with their target audience. Each student group designed their own test protocol and recruited test participants, then presented a report of the usability test to the key stakeholders of the MRU library.

Outcome

The participants found a number of aspects, both positive and negative, while completing the tasks that were provided. Certain issues such as the site’s inflexibility, complexity, and lack of customizability made it difficult for some to complete a certain task. Most of the participants’ issues stemmed from not being able to find content to complete a task easily. Others have found the website to have high learnability, even for some participants who have never used the website. Both the moderators and participants had a lot to take away from the usability tests. Participants learned the importance of clean data, the expectation for product or service improvement, the value and caveats with self-reported data, and the interaction between the participant and the moderator. The moderators mostly learned participant recruiting strategies.