In the ever-evolving landscape of mobile applications, ensuring a seamless user experience is critical, especially for applications catering to users with diverse needs. The iterative usability testing process we conducted for the UAM Go App stands as a testament to the value of continuous refinement, user feedback, and adaptability.
Overview
The UAM Go app is designed to enhance campus navigation for students with diverse needs and accessibility requirements, offering features such as building and room information and real-time alerts. Our team conducted four usability tests in collaboration with a major university involving students with self-reported disabilities to refine the app's design solutions. This article provides an overview of the initial four sessions, highlighting the invaluable insights gained through iterative testing. The rapid improvements made during these stages underscore the essential role of the iterative process in enhancing the app's usability even before its official launch.
Usability test format
The testing format for all four sessions was analogous:
- Preparation: Preparing the research goal, analysis, setting selection criteria, recruitment of participants in collaboration with the University, pilot testing.
- Introduction: Each participant was given a short introduction to the testing session and what would be involved.
- Accessing the app: Depending on the project stage, a prototype of the app or a testing instance of the app was downloaded or loaded to the participant's personal device.
- Tasks: Participants were asked to perform 3-5 tasks that were provided with clear instruction, with a "think-aloud" protocol.
- Summary interview: The session concluded with a short interview or conversation to discuss the participant's impressions and address any further questions.
Each testing round comprised of a diverse group of participants, including individuals with vision, hearing, mobility and cognitive disabilities. This diverse participant pool allowed for a comprehensive assessment of the app's usability across various accessibility needs.
User testing #1 - Onboarding & Main views
The research objective of the first session was to evaluate app onboarding and the main views of the app.
Main observations: Participants found the user interface visually appealing, clear, and intuitive. They expressed excitement about the app's potential usefulness and eagerly inquired about the release date. The identified challenges revolved around onboarding, where certain terms and distinctions proved confusing.
Recommendations: Simplify/reduce steps in the onboarding process for clarity, addressing the language in instructions to ensure more universal understanding.
Outcomes: Re-design and implementation of onboarding changes and main views completed and were deployed before the next testing session.
User testing #2 - Screen readers
In the second session, the objective was to evaluate how people with visual impairments using screen readers would perceive and use the basic features and interface of the app.
Main observations: The onboarding changes implemented after the first round of tests significantly improved the user experience.
We identified some content and design issues that interfered with the users' understanding and use of the app. For example, some phrases shown in the settings screen and ambiguity in buttons like "Map options" caused some confusion. No issues were detected in how the screen reader interacted with the app aside from minor bugs associated with the development stage.
Outcomes: The UAM Go app received positive feedback, with users praising its utility and expressing surprise at its high level of accessibility. The tests affirmed the correctness of solutions adopted after the initial round, identifying only minor issues that can be promptly addressed.
User testing #3 - Navigation & Maps
In the third session, the objective was to asses the navigation functionality of the app, as well as verify some design changes that had already been implemented following the previous two tests. Six participants with mobility or cognitive impairments were recruited. Tasks covered onboarding, route search, following navigation instructions, and feedback on alerts.
Main observations: Onboarding improvements were successful, with no reported issues in the current round. Participants noticed and acknowledged evacuation alerts, demonstrating effective communication. Users navigated the app efficiently, indicating a clear understanding of its element.
One of the key issues identified was that users expected GPS-based navigation, causing confusion when the app's functionality differed (it is unable to locate users inside buildings). The interface's resemblance to GPS navigation led to misinterpretations and user disorientation. Furthermore, routes often consisted of short, perpendicular segments, which felt unnatural, especially in open spaces. Users preferred instructions referencing landmarks rather than distances, finding the latter challenging.
Recommendations: Firstly, some changes to the routing and navigation experience were recommended to help align the path with more natural movement, for example simplifying the paths and emphasizing landmark-based instructions. Secondly, in order to address the expectations of GPS-like navigation, we suggested some instructional popups and changes in map design to better differentiate UAM Go from traditional navigation apps.
Outcomes: Updates to routing instructions, visual representations of current route segments, optimization of routes to follow more natural movement. Implementation of pop informing users to "break" their mental model.
User testing #4 – Checking final performance
In the fourth and last session, the objective was to asses the final and complete user experience at the end of the development process. The emphasis was especially put on the quality of the navigation functionality after all the changes following the third round testing session. Nine participants with vision, hearing, mobility or cognitive impairments were recruited. Tasks covered route search, following navigation instructions, and feedback on Campuses menu and organization of its content.
Main observations: The key challenge observed in the third round of testing was to break the mental model of the users, who expected GPS-based navigation. Changes in the graphic representation of the route and adding an informative pop-up added up to make a great improvement. Participants demonstrated full understanding of how the navigation works and had no problems following the route. All the participants successfully reached the destinations required in the task given.
We observed slight differences in how people with vision loss, using a screen reader, navigated with the app. They focused solely on the list of steps with no interest in using the "next" button. Another topic checked was Campuses menu. Participants valued the amount of handy information, although found the representation of it slightly overwhelming.
Recommendations: As it was the final round of the usability testing, we divided the recommendations into two categories – small improvements to be implemented immediately and before the launch of the app and bigger topics to be considered in the future. The first category included better organization of the content in the Campuses menu and minor technical bugs. Bigger topics, such as potential customization tool for the navigation, require more research and therefore were left out for the future, if the client decides to further develop the product.
The power of iterative testing
The outcomes of each testing round are a true testament to the value of iterative usability testing, even on limited budgets or simple prototypes. No matter the stage of development, there are worthwhile insights to be gathered from user feedback. The primary success outcomes of our testing were:
- Major refinement of the onboarding process: The iterative testing process allowed the team to rapidly address specific usability issues, particularly in the case of the onboarding process and clarifying confusing terms. We were able to implement improvements and by the end of the second testing session, most users understood the app's purpose much more clearly and we could observe in positive effect on the participants' experience.
- Insights into user expectations: The expectation of users for an experience similar to popular navigation apps highlighted the importance of aligning OR disconnecting the app's functionality with user mental models. This insight led to modifications in the app's interface to teach users how the app really works and guide users on the correct path.
- Real-time user feedback: Participants' real-time feedback during the iterative rounds provided invaluable insights into their expectations, challenges, and preferences. This direct interaction empowered the development team to make informed decisions for a more user-centric application for many different topics and design choices.
- Effect on the client's approach: Through iterative user testing, we unearthed unexpected usability issues for the client. These revelations compelled them to question and refine their original assumptions, ultimately leading to more informed and strategic product decisions.
Conclusions
The UAM Go App's iterative usability testing journey exemplifies the power of user feedback in shaping a more inclusive and user-friendly application. By recognizing challenges early in the development process, the team has been able to make targeted improvements, ultimately ensuring a more positive user experience. This iterative approach not only optimizes the app before its launch but also sets the stage for ongoing enhancements based on user insights, reinforcing the app's adaptability and responsiveness to user needs.
