It's all good and well having a product designed and developed, but without testing it's as good as not having built a product at all.
Testing is a way to make sure that your product can withstand traffic, that it is accessible to a multitude of user types - and in it's most basic way, we want to make sure that your product has a logical flow to it (usability).
"But Viv," I hear you say in an exasperated voice, "Where do I even begin? There is so much to look at and go through - I feel overwhelmed." Never fear darling, I got you.
First and foremost, we must develop a structured plan for each test instance, including objectives, methods, expected outcomes, and metrics for measurement. Without a plan, you may find yourself spending months and months going in a direction that has no end. Let's look at an example of a structured plan.
Structured Plan Example
In this examples we have designed and developed an a B2C e-commerce app for a gardening store in our local area. We have met with the owners multiple times and we understand what they need out of our service; they want to be able to sell plants to their usual customers as well as reach a little further and deliver plants slightly outside of the local area - hopefully expanding their commercial reach.
1. Test Objectives
Primary Objective: Evaluate the overall usability of the gardening store website, focusing on ease of navigation, efficiency in finding and purchasing products, and user satisfaction.
Specific Objectives:
Assess the intuitiveness of the site’s navigation structure.
Determine the effectiveness of the search function in helping users find specific plants or gardening tools.
Evaluate the checkout process for any potential barriers to completing a purchase.
Gauge user satisfaction with the visual design and content organization.
2. Methods
Test Participants:
10-12 participants who are regular online shoppers and have an interest in gardening.
Participants will be divided into two groups: novice gardeners and experienced gardeners.
Test Environment:
The test will be conducted in a controlled environment with participants using desktop computers and mobile devices to assess both the desktop and mobile versions of the website.
Remote usability testing tools will be used to track participants' actions and gather qualitative feedback.
Task Scenarios:
Task 1: Find a specific plant (e.g., “Fiddle Leaf Fig”) and add it to the cart.
Task 2: Search for gardening tools, compare prices, and choose the best option.
Task 3: Use the site’s filter function to narrow down plant choices based on light requirements (e.g., “low light”).
Task 4: Complete the checkout process using a promo code provided.
Task 5: Locate the “Plant Care Tips” section and identify a guide on how to care for indoor plants.
Data Collection Methods:
Screen recordings of user interactions.
Post-task questionnaires to assess user satisfaction.
Think-aloud protocol to capture users' thoughts and reasoning during tasks.
Observation and notes by the facilitator.
3. Expected Outcomes
Usability Improvements: Identification of specific areas where users encounter difficulties or inefficiencies (e.g., confusing navigation, unclear labeling, slow checkout process).
User Behavior Insights: Understanding how different user groups (novice vs. experienced gardeners) interact with the website, revealing potential areas for personalization or targeted improvements.
User Satisfaction Levels: Insights into how satisfied users are with the website’s design, functionality, and content, providing a basis for enhancements.
Task Completion Rates: Determining how easily and efficiently users can complete essential tasks on the site, influencing future design and development priorities.
4. Metrics for Measurement
Task Success Rate: Percentage of participants who successfully complete each task without assistance.
Task Completion Time: Average time taken by participants to complete each task.
Error Rate: Number and types of errors encountered during task completion.
System Usability Scale (SUS) Score: A post-test questionnaire to quantify overall usability.
Satisfaction Ratings: Average satisfaction score from post-task and post-test questionnaires, using a Likert scale (1-5).
Navigation Efficiency: Number of clicks or steps required to complete a task compared to an optimal path.
Abandonment Rate: Percentage of users who start but do not complete a task, especially during the checkout process.
A structured usability testing plan provides a clear roadmap for evaluating the gardening store website's effectiveness and user experience. By analyzing the data collected from these tests, actionable insights can be derived to optimize the website, ensuring a seamless and satisfying experience for all users.
Secondly, it is important to remember that testing (and designing, and development) is an iterative process - and that might feel repetitive - but if you're not iterating, you're submitting what you think the first version of anything you make is perfect - and that simply is very unlikely to be true. So be prepared to go back and forth with your team and your product, and learn to enjoy the idea that you will be revisiting your creation over the next months (or years) with the express intent to add or toss out parts that no longer serve the UX/UI or business goals.
Awesome! Now we are getting somewhere. You're ready to look at the testing types and the sub-categories of those tests.
Usability testing
Task Completion Tests
Observe users as they complete specific tasks using the features of your product. Note where they struggle or get confused. You can use a product like Hotjar for this part, as it allows you to record users interactions, track sticking points, U-turns and rage clicks.
Think-Aloud Protocol
Sitting in a physical group with users and having them verbalize their thoughts as they interact with the app to understand their thought process and pain points. Oftentimes users can give a lot away by how they are physically reacting to an app, which, if your ego is strong enough, can help you repair your broken product.
Eye-Tracking Studies
Sometimes users cannot verbally express their habits - because they simply do not know their own selves well enough to say. Use eye-tracking technology to see where users focus their attention, which can reveal how intuitive the product's layout and design are.
Technical Performance Testing Load Testing Assess how the product performs under high usage conditions. What happens if there are 10, 100 or 1000 people all landing on your site at once? Can it handle those types of volumes? What happens when it does?
Response Time Testing Measure the time the product takes to respond to user inputs. If response times are too slow, you may end up with frustrated users pulling frowny face at their device and moving off to find something that responds better.
Reliability Testing
Evaluate how reliably the product performs its intended functions over time. Is the user experiencing the same experience every single time, or are they expecting mixed results from your app? (hint: they should not be expecting mixed results - that's bad).
Accessibility Evaluation
Accessibility Testing
You want everyone to be able to enjoy your creation - so ensure the product is accessible to users with disabilities (like colour blindness), checking for compliance with standards like the ADA (Americans with Disabilities Act) or WCAG (Web Content Accessibility Guidelines).
User Experience (UX) Evaluation
Heuristic Evaluation Conduct a heuristic evaluation to assess the app against established usability principles. If you're unsure about what that means - look below at Nielson's Heuristics.
Nielsen’s 10 Heuristics
Visibility of system status
The system should always keep users informed about what is going on through appropriate feedback within a reasonable time.
Match between system and the real world
The system should speak the users' language, with words, phrases, and concepts familiar to the user, following real-world conventions.
User control and freedom
Users often choose system functions by mistake and need a clearly marked "emergency exit" to leave the unwanted state without having to go through an extended process.
Consistency and standards
Users should not have to wonder whether different words, situations, or actions mean the same thing. Follow platform conventions.
Error prevention
Even better than good error messages is a careful design that prevents a problem from occurring in the first place.
Recognition rather than recall
Minimize the user’s memory load by making objects, actions, and options visible. The user should not have to remember information from one part of the dialogue to another.
Flexibility and efficiency of use
Accelerators—unseen by the novice user—may often speed up the interaction for the expert user such that the system can cater to both inexperienced and experienced users.
Aesthetic and minimalist design
Dialogues should not contain information that is irrelevant or rarely needed. Every extra unit of information in a dialogue competes with the relevant units of information and diminishes their relative visibility.
Help users recognize, diagnose, and recover from errors
Error messages should be expressed in plain language (no codes), precisely indicate the problem, and constructively suggest a solution.
Help and documentation Even though it is better if the system can be used without documentation, it may be necessary to provide help and documentation.
User Satisfaction Surveys Gather feedback from users on their satisfaction with the app using surveys or questionnaires. You can make use of something like Hotjar or a myriad of other tools that offer a feedback option. A great idea is a "roast us" banner on your site that allows users to be brutally honest but have a little fun with it - unlike the traditional "feedback" survey which can feel dry and tedious.
User Experience (UX) Evaluation
Field Testing Test the product in it's actual deployment environment to see how environmental factors (like lighting, noise, etc.) affect it's usability. This might lead you to consider things like dark and light mode, changing the type of audio you use, or even reconsider the haptics.
Ergonomics Testing
Evaluate the physical ease of use, considering factors like height, screen angle, and touch interface responsiveness. There are so many screen sizes these days - some phones are impractical to operate with one hand - does your app require one handed use, or for the user to stretch their lil thumbs to the top of the screen?
Security Testing
Penetration Testing Check for vulnerabilities that could be exploited by malicious parties. Without the proper measures in place, you could be exposing them to malware or phishing software that could get both them and you in trouble and you do absolutely not want to be responsible for your user's bank account being drained because of your poor security measures.
Data Privacy Compliance Testing Are you collection users information? Banking details? Well, these days people's information is worth it's weight in Bitcoin. Nobody wants to receive unsolicited calls or mails. Ensure the app complies with relevant data protection regulations like POPII, GDPR or HIPAA.
Feedback Loops
Social Media Monitoring Besides what we have already mentioned above about questionnaires and roasting prompts, track what users are saying about the app on social media platforms for unfiltered feedback.
Analytics and Reporting
Usage Analytics Analyze usage data to understand how frequently different features are used and identify patterns. You can use tools like Google analytics, or Mixpanel to get to the core of the user's behaviors.
Error Reporting Monitor error logs to identify and address technical issues promptly.
Comparison Testing
Benchmarking Compare the product's performance against similar units or industry standards to see where it stands relative to competitors or best practices. It is in any case a good idea to keep tabs on your competitors to see what they are up to.
Testing and Exploration
Avoid overwhelming with too much information. Provide links to articles and references for in-depth information, and maintain an easily navigable system for others to understand ongoing work.
Knowledge Base Develop a knowledge base where articles and references are organized and easily searchable. You want things like handover and scaling to be as smooth a transition as possible. Consider doing SOP's and keeping well maintained documentation.
Executive Summaries Provide executive summaries for each major piece of documentation, highlighting key points and decisions. This will make it easy for people to scan through what you have done, and where the product stands in relation to that.
"Gosh Viv, that is a lot of information to process!". Yes darling, it is, but you know what - by using these or a selection of these approaches, your research and iterations are going to touch on all the major points which can make or break your product in the end.
It will definitely give you a very holistic approach to ensuring you are following the best practices throughout your design and development process - which will give your product the best chances at success, and I want so much for you to succeed in all that you do.
Comments