UX Firm

Common Usability Testing Mistakes and How to Avoid Them

Fri, 10 Apr 2026 16:52:26 GMT

Usability testing is one of the most effective tools in creating products that truly resonate with users. But getting the most out of usability testing requires more than just observing users interact with your product. Mistakes during the planning or execution phase can skew your results, waste resources, and even lead to faulty design decisions. Teams that want to avoid these issues often benefit from expert usability testing services that improve recruiting, study design, moderation, and reporting.

This blog highlights some common usability testing mistakes and, most importantly, how you can steer clear of them. Whether you’re new to usability testing or a seasoned pro, avoiding these pitfalls with the help of usability research experts , will ensure your product development process remains efficient and user-centered.

1. Selecting the Wrong Participants

The Mistake:
Failing to recruit participants who accurately represent your target user group is a common blunder. This could mean testing with people who are too familiar with your product or, conversely, individuals who are completely outside your audience. A mismatch here leads to results that don’t translate into meaningful improvements for real users.

How to Avoid It:
Define precise user personas before recruiting participants. Think about the demographics, behaviors, needs, and goals of your typical users. Use appropriate recruiting tools and services to match participants to these criteria. If your product is niche, be ready to go the extra mile—such as partnering with industry professionals or leveraging social media groups—to find the right users.

Impact Example:
Imagine testing an app intended for tech novices but selecting participants from a tech-savvy audience. Their feedback might overestimate the usability of complicated features, leaving your actual users struggling post-launch.

2. Poor Test Planning

The Mistake:
Jumping into usability testing without a solid plan is a recipe for disaster. This often includes vague objectives, an unstructured approach, or a testing environment ill-suited to the product. Poor preparation can result in wasted sessions, unhelpful feedback, and inconclusive insights.

How to Avoid It:

Approach usability testing as a structured experiment. Begin with clear objectives—what are you specifically testing for? Is it navigation pathways, feature comprehension, or the overall flow? Develop detailed test scripts that include key tasks to observe, but allow room for users to explore naturally. Ensure tools, environments, and prototypes are fully functional beforehand to avoid interruptions during testing.

Impact Example :

A team once tested an e-commerce website’s checkout flow but forgot to prepare mock user accounts. The session stalled midway through, frustrating participants and yielding no usable data.

3. Ignoring Non-Verbal and Contextual Cues

The Mistake:

Relying solely on task completion data or verbal feedback can lead to missing important insights. Users might hesitate to voice their confusion or frustration openly, especially if they are in an unfamiliar testing environment.

How to Avoid It:
Pay attention to non-verbal cues like hesitation, facial expressions, or repeated actions during usability tasks. Ask follow-up questions to clarify their behavior, like “What were you expecting to happen here?” or “How did that feel for you?” Combining verbal and behavioral insights will give you richer perspectives.

Impact Example:

During testing for a wearable health monitor, subtle signs—like users struggling to buckle the strap—went unnoticed. The oversight resulted in a mass recall after complaints post-launch.

4. Focusing Only on What Users Say

The Mistake:
While user feedback is valuable, solely relying on what participants say can be misleading. Users may not fully understand their preferences during the test or feel compelled to give positive feedback.

How to Avoid It:
Analyze actions alongside spoken feedback—what the user does is just as important, if not more so, than what they say. Gather objective data, such as task completion rates, times, and clicks, to validate user insights.

Impact Example:
A startup redesigned their app’s onboarding process based on exclusively positive feedback from tests, only to receive complaints post-launch about its length. Objective da

5. Skipping a Pilot Test

The Mistake:
Skipping a pilot test is a surefire way to miss flaws in your methodology. This often leads to moments during live tests where errors muddy the results—e.g., unclear instructions or technical issues with the tools being used.

How to Avoid It:
Before rolling out usability tests to participants, conduct a pilot session. This helps identify and iron out any logistical or procedural kinks in the test. Treat the pilot run like a dress rehearsal to ensure everything flows smoothly during the real thing.

Impact Example:
Test scripts for a restaurant app failed to account for missing menu categories in one usability test. Had a pilot run been conducted, the team could have corrected the gap in advance.

6. Ignoring Broader Context

The Mistake:
Usability tests conducted in a highly controlled environment can ignore how external factors impact the user experience. For instance, distractions from the user’s daily life or device limitations may play a significant role.

How to Avoid It:
Whenever possible, complement in-person usability tests with remote ones. Remote tests, conducted in a natural setting, often reveal new challenges users may face in real contexts. Also, simulate real-world environments during in-person tests when feasible—such as asking participants to use their mobile phones or putting them under realistic time constraints.

Impact Example:
A navigation app was deemed flawless in a lab environment. However, actual users found its interface too cluttered when used on-the-go, like while driving or walking. Testing in real-life scenarios would have flagged the problem early.

7. Failing to Act on Results

The Mistake:
Collecting insightful data but failing to translate it into action is one of the most frustrating pitfalls of usability testing. Sometimes organizations are too quick to dismiss user feedback or fail to align development timelines with testing outcomes.

How to Avoid It:
Integrate usability test results into decision-making processes immediately. Prioritize findings based on severity and frequency of issues, and turn them into actionable steps for your development team. Conduct retests after making changes to validate improvements.

Impact Example:
A retailer developed a mobile app but ignored user complaints about unclear promotions. Sales suffered until the issue was addressed a year later—a costly delay.

Closing Thoughts

Usability testing is a crucial step toward creating user-centric products. However, the effectiveness of your efforts hinges on thoughtful planning, execution, and follow-up. By avoiding these common mistakes—like recruiting the wrong participants, skipping a pilot test, or neglecting to act on insights—you’ll be better equipped to gather actionable findings that truly enhance your product.

Every test is an opportunity to learn. By focusing on your users’ needs and behaviors while steering clear of these pitfalls, you can deliver products that not only meet but exceed expectations. Ultimately, it’s about creating solutions that make users’ lives easier—and isn’t that the goal of great design?

How UX Firm Can Help

At UX Firm, we specialize in guiding teams through every step of the usability testing process, from identifying user needs to conducting usability testing and reporting the findings with action-based recommendations for continued improvements in user experience.

Schedule a free consultation with UX Firm today. Together, we’ll help you avoid common usability testing mistakes for better results.

Usability Testing FAQ's

Related Blogs About Usability Testing

Choosing Between Remote and In-Person Usability Testing: What Works Best?

Mon, 13 Oct 2025 15:58:58 GMT

When it comes to creating products that resonate with users, usability testing plays a critical role. But once you’ve decided to evaluate your product’s usability, another question arises—should you conduct this testing remotely or in person? Each method has its unique advantages and drawbacks, and the right choice depends on your specific project requirements.

This article will guide you through the pros and cons of remote and in-person usability testing, key factors to consider when making your choice, and tips for success in both approaches.

Whether you choose remote usability testing or in-person testing, the goal of usability testing is to evaluate a product by observing how real users interact with it, identifying pain points and areas for improvement. The insights you gain from usability testing ensure the product is intuitive, functional, and satisfying to the user. How you conduct this testing—remotely or in person—can significantly influence your results.

Remote Usability Testing

Remote usability testing involves participants completing tasks from their own location while researchers observe and collect data through tools like video conferencing, screen sharing, or automated usability platforms. This method has gained popularity, especially with the rise of distributed workforces and advanced testing tools.

Advantages of Remote Usability Testing

Wider Reach : You’re not limited to participants in a specific geographical area. Remote testing allows you to recruit participants from anywhere, ensuring a more diverse and representative sample.
Cost-Effective : It eliminates travel costs, venue expenses, and logistics associated with in-person testing.
Faster Turnaround: Without the need to coordinate physical meetings, remote testing can often be set up and executed more quickly.
Natural Environment : Participants use the product in their everyday surroundings, offering insights into how it fits into their real-life context.

Disadvantages of Remote Usability Testing

Limited Interaction : Researchers may find it harder to observe non-verbal cues such as body language or facial expressions, which can be valuable indicators of user frustration or satisfaction.
Tech Dependence : Remote sessions rely on stable internet connections, functioning hardware, and usable software. Any technical glitches can disrupt the testing process.
Lower Control : The researcher has less control over the testing environment, which means external factors could influence participants’ behavior.

In-Person Usability Testing

With in-person usability testing, participants come to a controlled setting like an office, lab, or designated venue to perform tasks while researchers observe directly.

Advantages of In-Person Usability Testing

Rich Insights: Researchers can pick up nonverbal cues and directly ask followup questions to dig deeper into the user’s thoughts and reactions.
Controlled Environment: Testing conditions are standardized, minimizing external factors that might influence results.
Technical Intervention: If participants struggle with the setup or encounter technical difficulties, researchers can step in immediately to resolve issues.
Stronger Researcher-Participant Connection: Building rapport face-to-face can make participants feel more comfortable and open.

Disadvantages of In-Person Usability Testing

Higher Costs : Renting a venue, transporting participants, or arranging accommodations can significantly increase expenses.
Time-Consuming : Coordinating schedules, setting up the location, and managing logistics take time.
Geographical Limitations : You’re restricted to participants who can travel to the testing site, potentially limiting the diversity of your sample group.

Factors to Consider When Choosing a Method

Before choosing between remote and in-person usability testing, assess the following factors within the context of your project:

Budget : If you’re working with limited funds, remote testing is usually the more affordable option.
Timeline : Need quick results? Remote testing often has a faster setup phase compared to in-person testing.
Participant Availability : Are your target participants spread across locations, or do they have limited time to travel? Remote usability testing may be the better option.
Nature of the Product :
For digital products like apps or websites, remote testing is often sufficient.
For hardware or physical products, like medical devices or consumer products, in-person testing is typically necessary to observe how users interact with the tangible elements.
Type of Insights Needed: If behavioral cues, such as hand movements or facial reactions, are critical to your study, in-person testing is more effective. However, if your focus is mainly on task completion or navigation, remote testing may be sufficient.

Scenarios Where Each Method Shines

When to Choose Remote Usability Testing

Evaluating a global product that targets users across different regions and cultures.
Testing a mobile app or software interface where all interactions occur digitally.
Getting quick feedback on a prototype or feature of a product in the early design stages.

Example : A SaaS (Software as a Service) startup rolls out a new interface for their product and seeks feedback from diverse users across the globe. Remote testing allows them to gather input rapidly without logistics concerns.

When to Choose In-Person Usability Testing

Testing physical or hardware-based products that require hands-on interaction.
When studying specialized medical devices or equipment that involves complex user workflows.
Validating key design decisions for high-stakes products, such as those in healthcare or aviation.

Example : A medical device company testing a new wearable device uses in-person sessions to observe how easily users can put it on and operate it. They also monitor any confusion with physical buttons or user instructions.

Tips for Effective Testing in Both Settings

Remote Usability Testing

Use Reliable Tools : Invest in software that facilitates screen sharing, recordings, and real-time feedback, such as Zoom or Teams, or subscribe to one of the remote testing platforms like UserTesting or Lookback.
Prepare Instructions : Ensure participants receive clear, step-by-step guidance to reduce confusion or setup delays.
Test the Tech First : Verify that all tools work seamlessly before the session begins. Conduct a trial run with a colleague.

In-Person Usability Testing

Create a Comfortable Setting : Make sure participants feel at ease in the testing room. A relaxed participant is more likely to provide genuine feedback.
Record Everything : Use cameras and screen-recording tools to capture every detail, enabling further analysis later.
Plan for Contingencies : Allow extra time for potential delays or unforeseen issues, particularly with physical prototypes.

What is the Best Approach?

Whether you opt for remote or in-person usability testing, the ultimate goal is the same—to gather valuable insights that make your product better. Remote testing offers scalability, speed, and cost efficiency, while in-person testing provides deeper insights and more controlled conditions.

The best approach often depends on the nature of your product, your available resources, and the questions you need answered. Remember, you don’t need to stick to one method exclusively—many projects benefit from combining both remote and in-person techniques for a holistic understanding of usability.

By carefully weighing your options and focusing on the needs of your users, you’ll be well on your way to delivering a product that’s as functional as it is delightful to use. If you need expert guidance to navigate these decisions, partnering with a user research agency can ensure you get the reliable data necessary for success.

Frequently Asked Questions

Related Blogs

The Science Behind Usability Testing: Why It Matters for Your Product’s Success

Thu, 18 Sep 2025 20:44:58 GMT

image source: https://measuringu.com/eye-tracking/

Creating products that users love isn’t just about good intentions; it’s about understanding their needs, behaviors, and challenges. That’s where usability testing comes in. At its core, usability testing is a powerful tool fueled by scientific principles and a data-driven methodology. It provides insights into how real users interact with a product, revealing opportunities for meaningful improvement. Regardless of your industry, mastering usability testing can make the difference between a product that succeeds and one that falls short.

Here, we’ll explore the science behind usability testing, its role in product development, and how it can drive your product’s success.

What is Usability Testing?

Usability testing is the process of observing real people as they use a product or prototype to complete specific tasks. The goal is to evaluate how easy it is for users to achieve their objectives and uncover any pain points in the process. This approach doesn’t rely on assumptions or opinions—it generates empirical data that you can act on.

The foundation of usability testing is rooted in cognitive psychology and human-computer interaction (HCI). It draws on principles such as user-centered design , which places the user’s experience at the heart of the product development process.

The Scientific Method Applied

At its core, usability testing follows the structure of the scientific method:

Define the Hypothesis: What assumptions do you have about how your product works? For example, do you believe users will intuitively find the checkout button on your ecommerce site?

Gather Users: Recruit a representative group of test participants. These participants should closely match your product’s intended audience to yield insights that are both relevant and actionable.

Observe and Collect Data: Have participants complete tasks while you observe, either in person or using screen-recording tools. Quantify metrics like task success rate, time to complete tasks, and error rates.

Analyze Results: Identify where users struggled, and determine why. Use subjective feedback and comments from your participants to understand their actions. Are instructions unclear? Is the navigation unintuitive?

Iterate and Retest: Use the findings to tweak your design, then repeat the process to verify improvements. This iterative approach ensures solution-focused innovation. It roots recommendations in science, rather than personal guesswork.

Why Usability Testing Matters

The success of any product hinges on its ability to meet user needs. If potential customers can’t intuitively figure out how to use it, they’re unlikely to stick around. Usability testing helps you uncover these barriers long before the product goes to market.

1. Identifying Real User Needs

Usability testing is grounded in real-world use and ensures that your assumptions about your users align with reality. People aren’t always predictable, and user interactions often highlight preferences or challenges you didn’t anticipate.

For example, consider a healthcare app for tracking medications. If some users struggle to understand the app’s notifications reminding them to take pills, the usability test reveals this issue. By improving how notifications are designed, developers ensure users are less likely to miss doses.

2. Improving Product Design

Some factors critical to user satisfaction simply can’t be predicted without testing. Far too often, products are built on what developers think works, rather than what the user actually finds functional.

Take the design overhaul of the zero-calorie sweetener, Splenda. During its initial usability study for packaging, users demonstrated confusion over how to tear open the packets cleanly. Armed with this finding, the company redesigned the tear strip, vastly improving user experience. The result? Higher customer satisfaction—and increased sales.

3. Enhancing User Satisfaction

Users who can complete tasks smoothly and efficiently are more likely to have a positive experience with your product. Positive experiences not only drive loyalty but also encourage users to become advocates for your brand. Usability testing pinpoints frustrations, allowing you to address them before they tarnish your product’s reputation.

Examples of Success Fueled by Usability Testing

Airbnb

Airbnb’s rapid growth wasn’t just fueled by a great idea—it was backed by deep usability research. Their team found that hosts struggled to understand how to upload photos, a crucial part of their listing setup. After simplifying this process, Airbnb improved host satisfaction and increased the number of listings, helping the platform scale.

Dropbox

Dropbox famously simplified the process of storing and sharing files. Early usability tests revealed that people were confused about how files synced to the cloud. To combat this, Dropbox introduced simple visuals and a drag-and-drop interface. This change became the backbone of their user-friendly appeal.

The Pitfalls to Avoid

While usability testing is instrumental to success, certain common mistakes can hinder its effectiveness:

Testing Too Late: Waiting until the final stages of development leaves little room for improvement. The earlier you incorporate usability tests, the easier it is to adapt and optimize your design.

Skipping Realistic Scenarios: Testing needs to simulate how users interact with the product in their everyday environment. Artificial or overly guided scenarios may leave blind spots.

Ignoring Small Sample Sizes: Even a small group of 5-7 participants can yield meaningful patterns. You don’t need hundreds of users to identify major pain points.

Failure to Act on Results: Insights are only as valuable as the actions they inspire. Make sure your team is ready to incorporate findings into tangible design changes.

Common Usability Testing Methods

Here are four widely used techniques that bring rigor and reliability to usability studies:

Moderated Testing
A researcher directly interacts with participants and moderates the session. This method allows for deeper insight into user thoughts but requires experienced facilitators to avoid introducing bias.
Unmoderated Testing
Participants complete tasks independently while tools record their screen activity and emotional responses. It’s cost-effective and good for observing unbiased behaviors.
A/B Testing
Participants compare two variations to determine which performs better. This method helps fine-tune specific elements, be it layout, navigation, or calls to action.
Heatmaps and Click Tracking
These tools visually map which areas of your app or website users spend the most (or least) time on. They’re invaluable for refining navigation and feature accessibility.

Wrapping it Up

The science of usability testing offers an objective lens to evaluate and improve your product. From mitigating risks to delighting customers, it equips developers, designers, and product managers with the insights needed to create user-friendly, successful products.

If there’s one takeaway, it’s this: Don’t treat usability testing as a nice-to-have—it’s a must-have. The more you invest in gaining user insights, the closer you’ll get to delivering a product that exceeds expectations. Whether you’re launching an app, designing a medical device, or building an everyday consumer product, usability testing paves the path to success.

Frequently Asked Questions

Human Factors Validation in a Global Market: Meeting IEC and FDA Standards

Fri, 12 Sep 2025 03:58:08 GMT

The design and development of medical devices require more than just technological innovation. For a device to succeed globally, it must not only meet the needs of its users but also comply with stringent regulatory standards. Among the critical aspects of medical device compliance is human factors validation. Understanding and addressing human factors ensures that devices are safe, intuitive, and effective for their intended users and environments.

When targeting a global market, manufacturers must balance the requirements of multiple regulatory bodies. Two key players in this landscape are the U.S. Food and Drug Administration (FDA) and the International Electrotechnical Commission (IEC). While both aim to ensure the safety and usability of medical devices, their standards and expectations for human factors validation require careful navigation. Let's explore why this process is essential, examine the differences and overlaps between FDA and IEC standards, and discuss best practices for achieving global compliance.

Why Is Human Factors Validation Crucial?

Human factors validation focuses on ensuring that medical devices can be used safely and effectively by real-world users. It considers how individuals interact with devices and aims to reduce the risk of use errors that could compromise patient safety.

For example, if a device has a poorly designed interface or unclear instructions, the likelihood of errors increases. These errors—whether inputting incorrect settings or using the device inappropriately—can have serious or even fatal consequences.

Regulators such as the FDA and the IEC mandate human factors validation as part of the development process to address these concerns. Failure to meet their standards can result in regulatory delays, market rejections, or product recalls—setbacks that could tarnish a company’s reputation and financial future.

FDA and IEC Standards for Human Factors Validation

The FDA and IEC share a commitment to user safety, but their approaches and guidelines on human factors validation differ in important ways.

FDA Guidance

The FDA’s primary guidance is outlined in its 2016 guideline, Applying Human Factors and Usability Engineering to Medical Devices . This guidance emphasizes the evaluation of reasonably foreseeable use errors and device usability under real-world conditions. Key points include:

Focus on Safety : The FDA prioritizes the identification and mitigation of use errors that could result in serious harm or compromised device effectiveness.
Summative Human Factors Validation Testing : Usability validation must demonstrate that the device can be used safely and effectively by intended users in the intended environments.
Documentation : Detailed documentation is required, including test methodologies, participant demographics, error analysis, and design modifications.

IEC Standards

On the global stage, the IEC’s 62366-1 standard , Application of Usability Engineering to Medical Devices, serves as the primary guideline. It shares many goals with the FDA’s approach but has some distinct features:

Lifecycle Perspective : The IEC standard focuses on usability engineering throughout the device’s lifecycle, encouraging continual improvements after market release.
Risk Management Alignment : The standard aligns closely with ISO 14971 , which addresses risk management in medical devices.
Broader Scope : While the FDA places a heavy emphasis on safety-critical issues, IEC guidelines adopt a more holistic view of usability, including user comfort and efficiency.

Overlaps and Differences

Both the FDA and IEC require rigorous usability testing, thorough documentation, and the identification of user risks. Where they differ is in the prioritization of safety versus overall usability and in the depth of focus on risk management integration.

For manufacturers, meeting one standard does not automatically satisfy the other. Devices intended for both U.S. and international markets must address the nuanced requirements of each regulatory body, which often means customizing validation protocols to ensure compliance on both fronts. However, it is generally recognized that compliance with the FDA guidance document with regard to the number of participants and test protocol will provide compliance with the IEC standard.

Best Practices for Human Factors Validation

Conducting human factors validation that satisfies both IEC and FDA standards can seem daunting. However, adopting best practices can help streamline the process and reduce regulatory hurdles.

1. Start Early

Human factors validation testing should not be an afterthought. Incorporate usability engineering from the earliest stages of device design—well before entering formal validation testing. Early formative testing with prototypes allows for iterative improvements that address usability concerns before they escalate.

2. Understand Your Users

Define your intended user groups and use environments with precision. This includes professionals like surgeons and nurses, as well as patients or caregivers who may not have medical training. Create detailed user profiles to understand the challenges, expectations, and skill levels of each group.

For example, if you’re designing a home glucose monitor, you must consider elderly users with limited dexterity and low technical proficiency. Testing the device with this demographic ensures it meets their specific needs.

3. Use Realistic Test Scenarios and Use Environments

Both standards emphasize testing under conditions that replicate actual use environments. Simulate environments and provide appropriate and relevant scenarios for users engaging with your medical device in such environments as hospital emergency rooms, outpatient clinics, or patients’ homes to observe how users interact with the device in real-world settings.

High-fidelity test environments are particularly important for the FDA, which values summative testing as evidence of safety. For IEC compliance, these scenarios also help identify usability gaps beyond immediate safety concerns.

4. Develop a Unified Testing Strategy

To satisfy both FDA and IEC requirements, create a usability testing protocol that addresses shared principles while tailoring certain elements for each guideline or standard. For instance:

Include risk management processes to align with IEC’s ISO 14971 integration.
Focus explicitly on safety-critical tasks to meet FDA expectations.

By combining these approaches, you avoid duplicating efforts while ensuring all requirements are met.

5. Document Everything

Thorough documentation is non-negotiable. Maintain comprehensive records of usability testing, including participant recruitment criteria, test scripts, observed use errors, and design modifications made in response to testing outcomes.

FDA documentation may prioritize safety-related outcomes, while IEC requires a broader usability engineering file. Preparing for both ensures readiness for audit and regulatory review.

6. Seek Expert Guidance

Navigating the intricacies of global standards can be overwhelming, especially for small or newly established companies. Partnering with consultants like UX Firm who specialize in human factors validation and regulatory compliance can ensure you’re covering all bases.

Final Thoughts

Human factors validation in the medical device industry is more than just a regulatory checkbox; it’s a critical step in designing products that users can trust. Balancing the requirements of both IEC and FDA requires a thoughtful approach, but the effort is well worth it.

By prioritizing usability, safety, and user needs from the start, manufacturers can achieve smoother regulatory approvals and, more importantly, deliver medical devices that make a positive impact worldwide. Whether you’re developing wearable technology, diagnostic equipment, or life-saving devices, committing to robust human factors validation sets the stage for success in a global market.

How UX Firm Can Help

Human factors validation isn’t just about ticking boxes for FDA clearance or IEC compliance—it’s a rigorous process that ensures safer, more effective medical devices. At UX Firm, we specialize in guiding teams through every step of this process, from identifying user needs to conducting usability testing and creating FDA/IEC-compliant reports.

Our hands-on expertise helps you design devices that are not only compliant but user-friendly—a winning combination for success in the medical device industry.

If you want to streamline your human factors validation process and ensure your device meets the highest safety standards for both FDA and IEC, schedule a free consultation with UX Firm today. Together, we’ll help you bring your medical device to market with confidence.

Related Blogs

Avoid FDA Setbacks with Effective Usability Testing

Fri, 11 Jul 2025 14:50:16 GMT

Navigating the regulatory landscape of medical devices can be a challenging endeavor. For companies aiming to bring innovative solutions to market, meeting FDA regulatory requirements is a critical milestone. But as anyone in the industry knows, even a minor misstep can lead to delays, increased costs, or even product rejection.

One of the most effective ways to avoid these setbacks? Incorporate usability testing into your development process. Usability testing not only ensures that your device meets user needs but also helps identify and mitigate risks that could raise red flags during FDA reviews. Here’s why usability testing is essential and how it can safeguard your path to approval.

Why Is Usability Testing Crucial?

Medical devices are often used in high-pressure environments, where both usability and safety are non-negotiable. The FDA recognizes this and requires manufacturers to demonstrate that their devices are safe and effective for the intended user and use environment.

Usability testing, as part of human factors engineering, focuses on understanding how real-world users interact with a product. This prevents assumptions about user behavior, identifies design flaws early, and ensures the device can be used safely and effectively.

By conducting comprehensive usability tests, manufacturers can gather critical data to meet FDA medical device requirements and minimize the risk of usability-related setbacks.

Avoiding Common Usability Pitfalls

Many FDA setbacks stem from user errors or design oversights that could have been caught earlier. Below are examples of common issues and how usability testing helps address them:

Confusing Interfaces
Imagine a medical device with a touchscreen interface that requires nurses to enter patient data. If the interface is overly complex or unintuitive, it could lead to input errors or delayed responses in critical situations.

How Testing Helps : Usability testing reveals areas where users struggle with the interface. By observing real users in action, you can simplify navigation, adjust button placement, or streamline workflows to create a more intuitive design.
Misinterpreted Instructions
Poorly written or overly technical instructions could result in improper use of the device, potentially compromising patient safety. For instance, if users misunderstand how to set dosage levels on an infusion pump, the consequences could be dire.

How Testing Helps : T hrough usability tests, manufacturers can validate whether instructions are clear and actionable. Iterating on phrasing or including visual aids ensures better comprehension, reducing the risk of misuse.
Physical Ergonomics
Devices designed without consideration for user comfort may fail to gain approval. Take, for instance, a wearable device with straps that are uncomfortable or difficult to adjust. Although the technology might work perfectly, poor ergonomics can alienate users and raise concerns with regulators.

How Testing Helps : Testing prototypes with end-users highlights discomfort or awkwardness, allowing designers to tweak materials, shapes, or sizes to better suit users’ needs.
Failure in High-Stress Scenarios
Many medical devices, such as defibrillators or ventilators, are used in emergency conditions where fast, accurate operation is critical. A design that performs well in controlled lab environments might falter under the stress of real-world scenarios.

How Testing Helps : Simulated environments or high-pressure scenarios in usability testing can expose flaws that don’t arise under normal conditions. It ensures the device meets the demands of its intended use environment.

Demonstrating FDA Compliance Through Usability Testing

The FDA has specific expectations when it comes to usability testing, outlined in guidelines such as Applying Human Factors and Usability Engineering to Medical Devices . Meeting these expectations requires a structured usability testing approach, often starting in the early stages of development. Here’s how you can align your efforts with FDA requirements:

Define the Intended Use and Users
Start by identifying who will use your device (e.g., healthcare professionals, patients) and the environments where it will be used (e.g., hospital, home use). This helps focus your usability testing on realistic scenarios.
Conduct Formative Testing
During the design phase, run iterative usability tests to evaluate early prototypes. This helps refine the design and resolve basic issues before they escalate into major problems.
Perform Summative Testing
Summative testing, often referred to as human factors validation testing, is conducted later in development to validate the device’s safety and effectiveness under realistic conditions. Data gathered here plays a vital role in FDA submissions.
Document Findings Thoroughly
The documentation of your usability testing process is as important as the tests themselves. Include detailed records of methods, participant demographics, test results, and the actions taken to address any issues. This evidence demonstrates your commitment to meeting FDA expectations.

Looking Ahead

Effective usability testing isn’t just about avoiding FDA setbacks; it’s about designing better devices that meet user needs and improve patient outcomes. By focusing on users from the start, you not only reduce regulatory risks but also create products that stand out in a competitive market.

Whether you’re developing a wearable monitoring device, a diagnostic tool, or a surgical robot, usability testing should be an integral part of your process. It’s not a step to take lightly—it’s an investment in your product’s success and the trust of the people who will use it.

Take the time to prioritize usability testing. The effort you put in now can save months, or even years, of delays down the road—ensuring that innovative medical devices reach those who need them most.

Frequently Asked Questions

Top 5 UX Research Methods to Boost Ecommerce Sales

Fri, 06 Jun 2025 15:33:19 GMT

Creating a seamless and engaging online shopping experience is no easy feat, but it’s essential for driving ecommerce sales. A good user experience (UX) can mean the difference between a one-time visitor and a loyal customer. To achieve this, successful ecommerce businesses rely on UX research to better understand their users’ needs, behaviors, and pain points.

By applying the right UX research methods, you can uncover invaluable insights and make data-driven decisions to enhance your online store. Below, we’ll explore five effective UX research methods that can help you boost ecommerce sales.

1. User Interviews (IDIs)

User interviews, also called in-depth interviews (IDIs), are one of the most straightforward and impactful ways to learn about your target audience. This qualitative research method involves having in-depth conversations with users to uncover their motivations, challenges, and expectations.

Why It’s Effective:

You gain real, firsthand insight into users’ decision-making processes.
It helps identify unmet needs or frustrations that users may not even realize they have.
You can gather specific suggestions for improving your site.

For example, if you run an online clothing store, user interviews might reveal that customers struggle to determine their correct size. Armed with that insight, you can introduce a size guide or virtual fit tool, which could improve conversion rates dramatically.

Pro Tip:

Be sure to prepare a facilitator’s guide that includes open-ended questions that you can use to engage the user in conversation about their experience. Then, actively listen. Avoid leading users toward specific answers to ensure your findings are unbiased.

2. Usability Testing

Ecommerce usability testing is a hands-on method to evaluate how easily users can navigate your website or mobile device, such as finding a product, adding it to their cart, and checking out.

Why It’s Effective:

It identifies pain points in real-time, such as confusing navigation or slow-loading pages.
You observe natural user behavior as they interact with your site.
Fixing usability issues directly leads to fewer drop-offs and abandoned shopping carts.

For example, if users struggle to find a “filter by price” option during testing, you’ll know it’s time to make that feature more prominent. Improving usability can streamline the shopping experience and encourage users to follow through with purchases.

Pro Tip:

Conduct usability tests on desktop, tablet, and mobile versions of your site to ensure a consistent experience across devices.

3. A/B Testing

A/B testing, or split testing, involves comparing two versions of a webpage (Version A and Version B) to see which performs better. This quantitative research method offers concrete data on what resonates most with your audience.

Why It’s Effective:

You can test everything—from button colors to headline copy—to optimize conversion rates.
It offers measurable results, such as click-through rates, bounce rates, or sales.
Small changes inspired by A/B testing can lead to big improvements in performance.

Say you’re considering changing the color of your “Add to Cart” button. A/B testing can reveal whether users respond more positively to blue or green, helping you make a data-backed decision.

Pro Tip:

Test one variable at a time to ensure clear insights. Testing multiple changes simultaneously can make it difficult to identify what impacted the results.

4. Heatmaps

Heatmaps, which are generated using eye-tracking tools, provide a visual representation of where users are clicking, scrolling, or spending the most time on your website. With this tool, you can see exactly how people interact with your pages.

Why It’s Effective:

Highlights areas where users engage most and spots that are being ignored.
Helps identify distractions or confusing elements that prevent users from converting.
Makes it easier to optimize layouts by focusing on high-performing sections.

For instance, if your heatmap shows that users rarely scroll below the fold on your homepage, you might consider moving critical information or promotions higher up the page.

Pro Tip:

Combine heatmaps with usability testing session recordings for an even deeper understanding of user behavior on your site.

5. Customer Journey Mapping

Customer journey mapping outlines the step-by-step experiences users go through when interacting with your ecommerce site—from their first visit to the moment they complete a purchase (or abandon their cart).

Why It’s Effective:

Identifies friction points across the entire shopping experience.
Shows how users move from awareness to conversion, allowing for targeted optimizations.
Enables you to create a more cohesive and satisfying journey.

For instance, a customer journey map might reveal that many users abandon their carts at the payment stage due to limited payment options. Addressing this could significantly improve your checkout completion rate.

Pro Tip:

Include emotional insights in your map, such as how users feel at each stage, to create more empathetic and effective solutions.

Putting It All Together

By leveraging these five UX research methods—user interviews, usability testing, A/B testing, heatmaps, and customer journey mapping—you can create a more intuitive and user-friendly ecommerce experience. Each method uncovers unique insights, and together, they provide a comprehensive understanding of your users’ needs and behaviors.

The result? Fewer barriers to conversion, happier customers, and increased sales. Whether you’re fine-tuning a high-performing site or giving a struggling one a much-needed boost, investing in UX research is one of the smartest moves your ecommerce business can make. Take the time to prioritize your users, and you’ll pave the way for long-term success in a highly competitive market.

Curious about how to kickstart UX research for your ecommerce site? Reach out to UX Firm for a free consultation. We’d love to help you make positive changes to your customers’ shopping experience, and improve your bottom line!

Frequently Asked Questions

Want more practical insights? Check out these blogs.

Demystifying Human Factors Validation for Medical Devices

Fri, 09 May 2025 15:46:03 GMT

When you hear the term “human factors validation,” your first thought might be, “That sounds complicated!” But the truth is, it’s not as intimidating as it sounds. At its core, human factors validation is all about making sure medical devices are easy to use and safe for the people who need them most.

If you’re developing a medical device or just curious about what goes on behind the scenes, this process is a big deal. Why? Because it helps save lives, reduces errors, and ensures users—whether they're doctors, nurses, or patients—know exactly how to use the device properly. So, grab a cup of coffee, and let's break it down together.

What Exactly Is Human Factors Validation?

Imagine you’ve created a state-of-the-art medical device—something that could make a real difference. But here’s the catch. Even if your device is cutting-edge, if users can’t figure out how to use it easily or safely, it’s a problem.

This is where medical device human factors validation comes in. It’s the process of testing and refining your device to ensure that people can use it correctly, even in high-pressure or tricky situations. The goal is to identify and fix any potential use errors before your device hits the market. Plus, regulators like the FDA require you to prove that your device can be used safely and effectively by the intended users. That proof? It comes from human factors validation.

Why Is It Important?

Medical devices are often used in critical situations—think surgeries, emergencies, or at-home patient care. If a user misreads a button, has trouble understanding instructions, or struggles to operate your device, the results could be dangerous.

Human factors validation helps eliminate those risks by:

Pinpointing where users might get confused or make mistakes.
Enhancing device safety by addressing usability challenges.
Ensuring compliance with FDA guidelines, which are strict for good reason.

Long story short, it’s not just about checking a regulatory box. It’s about creating a device that works seamlessly for the people who need it most.

The Key Steps in Human Factors Validation

Now that we understand why it’s important, let's talk about how it actually works. The process can be broken down into a few clear steps:

1. Understand Your Users

Who will be using your device? This is where you step into the shoes of your potential users. Are they trained healthcare professionals? Older patients? Caregivers without medical experience?

Each user group comes with its own challenges. For example:

A surgeon might need a device that works quickly during high-pressure moments.
A senior patient might benefit from larger, clearer buttons.
A caregiver might need clear, no-nonsense instructions to operate the device safely.

Spend time researching your users' needs, environments, and skill levels. This understanding sets the foundation for everything else.

2. Design with Usability in Mind

Once you know your users, you can start designing a device that works for them . Think about how you can make your device intuitive. This might mean simplifying features, minimizing steps, or labeling buttons in a way that's crystal clear.

Remember, usability isn’t just about aesthetics—it’s about functionality. Every tweak you make now will help down the line when testing starts.

3. Conduct Usability Testing

Here’s where the real action happens. Usability testing puts your device in the hands of representative users in a controlled environment. You’ll ask them to perform specific tasks while you observe and take notes.

Pay close attention to:

Errors they make.
Questions they ask.
How confident they are using the device.

This is where potential issues come to light. Did they mistake one button for another? Was the display too small or confusing? Every insight you gather here helps refine the device and eliminate those pain points.

4. Analyze and Iterate

Once you have all your testing data, it’s time to analyze it. Look at where users struggled and figure out why. Do the instructions need rewording? Do you need to tweak the design?

Refining your device is an iterative process. You’ll go back to the drawing board, make improvements, and possibly test the device again to ensure the changes work.

5. Prepare for Validation

The final step is the actual validation—a formal process to demonstrate that your device is safe and effective for use by the targeted user groups. During validation, you’ll conduct rigorous testing under realistic conditions with real users.

You’ll focus on critical tasks—those moments where use errors could have serious consequences. The results will be compiled into an FDA-ready report, showing regulators that your device is as safe and intuitive as possible.

Making Medical Devices Better for Everyone

At the end of the day, human factors validation isn’t just about avoiding regulatory headaches. It’s about creating a product that truly works for the people it's meant to help. From reducing frustration to preventing dangerous mistakes, this process makes medical devices safer, easier to use, and more effective.

Take a moment to think about it—when users don’t have to second-guess how to use a device, they can focus on what truly matters: patient care and outcomes.

How We Can Help

If all of this still feels a bit overwhelming, don’t worry—you’re not alone. At UX Firm, we specialize in helping teams like yours navigate the human factors validation process. From identifying user needs to conducting usability testing and preparing FDA-compliant reports, we've got you covered every step of the way.

Want to learn more about how we can make validation less daunting? Reach out for a free consultation, and let's plan for the research that will demonstrate your medical device is safe and effective.

You’ve got the vision—we’ll help you bring it to life!

Frequently Asked Questions

Want more practical insights? Check out these blogs.

Small Budgets, Big Results: How Startups Can Maximize UX Research

Tue, 15 Apr 2025 15:38:00 GMT

Let’s face it—startups rarely have millions lying around to pour into UX research. But here’s the good news. You don’t need a massive budget to discover what your users want and improve your product’s user experience. With a bit of creativity and some savvy strategies, you can get big results without emptying your wallet.

If you’re a startup founder or a small business owner, this post is for you. Grab a cup of coffee, and let’s chat about simple, cost-effective ways to do UX research that makes a real difference.

Why UX Research Matters (Even for Startups)

First, a quick reminder of why UX research should be high on your startup’s to-do list. At its core, UX research is about understanding how people interact with your product. What do they love? What trips them up? What makes them abandon their shopping cart or struggle to find key features?

Getting answers to these questions early can save you time, money, and headaches later. The better your user experience, the more satisfied your customers, and the more likely they are to stick around or spread the word about your product. Now, who doesn’t want that?

Step 1: Start Small, Focus on the Essentials

When resources are tight, you want to focus on what’s going to make the most impact. That means thinking about your biggest questions and challenges. For example:

Are users getting stuck in your onboarding process?
Is it easy for them to complete the most important action (like making a purchase)?
Do they understand your product’s value right away?

Start small by honing in on these high-priority areas. You don’t need to tackle every page, button, or feature all at once. By narrowing your focus, you’ll get more actionable insights without feeling overwhelmed.

Step 2: Use Free or Low-Cost Tools

There’s no need to splash out on fancy software right away. Some of the best UX research tools out there are surprisingly affordable—or even free. Here are a few gems:

Google Forms or Typeform for quick surveys to collect user feedback.
Lookback or UserTesting for remote usability testing—it’s great for observing users in real time.
Figma or Miro for collaborative wireframing and user flow mapping.
Hotjar or Crazy Egg for heatmaps to see where users are clicking and scrolling on your site.

These tools help you gather insights without putting a dent in your budget.

Step 3: Conduct Remote Usability Testing

Speaking of usability testing, you don’t need to rent a lab or hire expensive moderators. Remote usability testing is a budget-friendly and flexible option for startups. Here’s how to do it in a way that works for you:

Define the tasks you want users to complete. For example, “Sign up for an account” or “Add a product to your cart.”
Use a tool like Zoom to watch participants perform these tasks from their own spaces. This approach gives you insights into how people naturally interact with your product.
Record the sessions so you can revisit key moments and share findings with your team.

Bonus tip: Hire an experienced moderator to conduct your sessions and observe live or from the recordings to understand your users' experience. Don't try to conduct these sessions yourself, as it takes skill and experience to do it right.

Step 4: Recruit Wisely (and Affordably)

Finding the right people for your UX research is crucial. But how do you do that without paying a fortune for a participant recruitment service? Here are some scrappy but effective ideas:

Ask your network. Friends, family, colleagues, or even people on LinkedIn can help. Just make sure they fit your target user persona.
Offer small incentives. You don’t need to shell out hundreds per user. Gift cards, discount codes, or even a small cash reward go a long way.
Go social. Post a call for participants on your social channels. Platforms like Instagram and Facebook are great for finding engaged users.

Remember, it’s not about gathering a huge group; even 5-8 participants can give you valuable insights.

Step 5: Get the Most Out of Your Research

Okay, so you’ve run your tests and gathered some data. Now what? The real magic happens when you take those results and turn them into actionable changes.

Identify trends. Are multiple people struggling with the same feature? Fixing that should jump to the top of your list.

Prioritize quick wins. Tackle easy-to-fix issues first for maximum impact.

Iterate and repeat. UX research is an ongoing process. Keep testing and improving to discover how your users interact with your product as it evolves.

Even if an issue seems small—like unclear button text—it can have a big impact on your users’ experience.

Why Small Actions Lead to Big Results

One of the most empowering things about UX research is how even small changes can make a huge difference. For instance, tweaking a confusing form can reduce frustration and improve conversion rates. Reorganizing your navigation can make it easier for users to find what they’re looking for.

And the best part? Each improvement adds up. Slowly but surely, you’ll create a product that’s not only functional but delightful to use—a surefire recipe for turning curious visitors into loyal customers.

No Need To Break The Bank

Startups may not have deep pockets, but when it comes to UX research, you don’t need to spend big to make a real impact. By focusing on high-priority issues, using affordable tools, and getting creative with participant recruitment, you can gather the insights you need to build a product your users will love.

Remember, it’s not about perfection—it’s about progress. With small, thoughtful steps, you’ll be amazed at how far you can go.

Frequently Asked Questions

Want more practical insights? Check out these blogs.

Human Factors Validation for FDA Medical Device Compliance

Thu, 13 Mar 2025 15:54:46 GMT

What Is human factors validation?

Human factors validation focuses on assessing how intended users interact with a medical device under expected conditions in simulated use. The FDA requires this process to confirm that the device can be used safely and effectively by the intended users without causing harm or errors.

It’s not about making assumptions—validation involves carefully designed usability tests where actual users perform critical tasks with the device. Their performance and any issues that arise form the basis for confirming that the device can be used safely and effectively by the intended users or if improvements are needed to the product’s design. If an iterative process has been used leading up to summative human factors validation testing, the results are more likely to confirm that users can safely use the device.

When it comes to medical devices, safety isn’t just a priority—it’s a legal and moral obligation. One critical process that ensures the safe and effective use of these devices is human factors validation . It’s not just about creating products that work; it’s about designing devices that work for the people using them. Most importantly, human factors validation plays a pivotal role in achieving compliance with FDA guidelines, making it a non-negotiable step in medical device development.

Whether you’re a medical device manufacturer or a product designer, understanding this process and its impact can save you from costly development missteps and bring life-saving devices to market faster.

Why does it matter for FDA compliance?

The FDA guidelines, particularly those outlined in the 2016 Applying Human Factors and Usability Engineering to Medical Devices and the corresponding IEC 62366-1 standard, emphasize the importance of human factors validation in ensuring public safety. The stakes are high—use errors in the context of medical devices can lead to severe injuries, adverse health outcomes, or even fatalities.

Here’s where human factors validation becomes crucial for FDA compliance:

Minimizes Use Error: By identifying the causes of misuse, companies can redesign devices to prevent errors, especially during critical operations.
Enhances Device Safety: Testing ensures that even under pressure or in unexpected scenarios, the device performs as intended.
Meets Regulatory Standards: FDA submissions require evidence of human factors validation, including detailed testing protocols and reports demonstrating that usability risks have been mitigated.

Skipping this step or approaching it inadequately often results in delays in the FDA review of pre-market notification submissions—or outright rejection.

The human factors validation process

Below is an overview of the essential steps in a typical human factors validation process. While it may vary depending on the product or company, every successful validation shares the same core elements:

1. Identify user needs and risks

The process begins with understanding your users, environments, and contexts of use. Are they healthcare professionals? Patients? Caregivers? Each user group will likely have unique challenges and expertise levels.

Identifying these needs is not just helpful but crucial. For example:

Does the user have limited dexterity or impaired vision?
Could the device be used in high-stress emergency situations?
What are the high-risk use scenarios that could lead to harm?

This stage also documents potential use hazards and the critical functions of the device where use errors could occur.

2. Conduct formative usability testing

Before human factors validation, many teams conduct formative testing to iterate on the product’s design. This isn’t required by the FDA but is highly recommended. Formative testing captures early usability insights and ensures that the product is heading in the right design direction.

3. Perform the validation study

The final step involves a formal usability test—the validation study. This is where your device goes through simulated use (or in some cases actual use) testing with representative users performing critical and non-critical tasks in a task flow.. The aim is to identify and understand potential use errors for the critical functions outlined earlier.

Key components during validation include:

Testing under Simulated Conditions: Users interact with devices in scenarios closely mimicking actual healthcare settings.
Critical Tasks Focus: Validation zooms in on steps or features that, if performed incorrectly, could lead to negative consequences.
Collecting Observable Data: Errors, close calls, and difficulties are documented for analysis, along with participants’ responses to post-scenario and post-test questions about their experience..

4. Analyze and report

The final step is to analyze the data and compile an FDA-ready human factors validation report. Findings from the validation study should clearly outline whether user needs have been met and if risks have been sufficiently reduced.

If any concerning errors or performance issues arise during validation, further iterations may be required to refine the design and go through another round of testing.

The benefits beyond compliance

While meeting FDA requirements is a compelling reason to invest in human factors validation, there are broader benefits for developers and manufacturers.

Reduced Development Costs: Spotting usability problems early prevents expensive redesigns later in the development process or after a device has been released to market.
Shorter Time to Market: A well documented human factors validation process speeds up FDA review, avoiding delays caused by incomplete submissions.
Higher User Satisfaction: Devices designed with human factors in mind are easier to use, which leads to better adoption rates and fewer negative reviews from end users.

Ultimately, the effort you invest in this process not only satisfies regulatory agencies but also strengthens your product’s position in a competitive market.

How UX Firm can help with your human factors validation

FDA medical device human factors validation testing isn’t just about ticking boxes for FDA clearance—it’s a rigorous process that ensures safer, more effective medical devices. At UX Firm, we specialize in guiding teams through every step of this process, from identifying user needs to conducting usability testing and creating FDA-compliant reports.

Our hands-on expertise helps you design devices that are not only compliant but user-friendly—a winning combination for success in the medical device industry.

If you want to streamline your human factors validation process and meet the highest safety standards required by the FDA, schedule a free consultation with UX Firm today. Together, we’ll help you bring your medical device to market with confidence.

Frequently Asked Questions

Want more practical insights? Check out these blogs.

How Usability Testing Can Transform Your Product Development Process

Fri, 07 Feb 2025 16:35:55 GMT

What is usability testing in product development?

Usability testing is a user-centered design technique where real users evaluate a product by completing specific tasks under controlled conditions. It’s a methodical way of uncovering design flaws, understanding user behavior, and gathering actionable insights to improve your product while it's in development.

The steps of the usability testing process

Usability testing isn’t just about handing someone your product and asking for feedback. It’s a carefully crafted process designed to extract valuable insights that drive meaningful change. Here’s what the process typically looks like:

1. Planning

Before testing begins, it’s important to define clear goals. What do you want to learn from the testing? Are you exploring a new feature, assessing navigation, or validating design choices? A well-planned protocol is key to ensuring the test aligns with your product development objectives.

2. Participant recruitment

Recruiting participants who match your target audience is crucial. At UX Firm, we use detailed screeners to pinpoint the right individuals for your product’s user profile. Whether it’s medical professionals testing a device or shoppers using an ecommerce site, the right participants ensure accurate, relevant insights.

3. Executing the test

Usability testing can be conducted in person or remotely, depending on your needs. Moderated testing sessions, whether virtual or face-to-face, allow an expert to guide participants, probe deeper into pain points, and ensure the test yields rich data for analysis. Tasks are designed to mimic real-world scenarios, providing a realistic gauge of how users interact with your product.

4. Collecting data

Throughout the testing process, detailed data is gathered—quantitative metrics like time on task and error rates, as well as qualitative feedback such as user comments and facial expressions. This data is essential for understanding what works, what doesn’t, and why.

5. Analyzing the results

Post-testing, it’s time to dig into the data. At UX Firm, our seasoned experts analyze findings to uncover patterns, identify pain points, and determine the root causes of usability issues. Then we translate these insights into actionable recommendations for improving your product.

The benefits of usability testing

1. Identifies user pain points early

One of the biggest advantages of usability testing is catching issues before they escalate. By identifying usability problems early, you can address them during development, saving time, money, and headaches down the line. For instance, discovering that users struggle to complete a vital action—like adding items to a cart or navigating a registration flow—allows you to fix these problems before launch.

2. Improves product design

Testing directly with users lets you see your product from their perspective, leading to better design decisions. You might find that a feature you thought would be a hit isn’t resonating, or that a small change drastically enhances clarity. These insights can refine your product, making it more intuitive and user-friendly.

3. Reduces development costs

Fixing issues during the design stage is exponentially cheaper than addressing them post-launch. Usability testing helps you allocate resources efficiently by guiding your design team to focus on what truly matters to users.

4. Boosts engagement and conversions

When users find your product easy and enjoyable to use, they’re more likely to stay engaged, recommend it to others, and convert into loyal customers. For B2B platforms, this could mean simpler onboarding; for ecommerce sites, higher sales. Usability testing ensures that your product isn’t just functional but delightful.

5. Supports regulatory compliance

For industries like healthcare, usability testing is indispensable. Human factors validation testing ensures that medical devices meet FDA and IEC standards, demonstrating their safe and effective use. Detailed testing and compliance reporting protect both users and your company.

Usability testing with UX Firm

At UX Firm, usability testing is at the heart of what we do. Our boutique consultancy is known for personalized attention and comprehensive solutions—from participant recruitment to expert moderation and evidence-based recommendation reports. We help businesses of all sizes, from Fortune 500 companies to startups, uncover the insights they need to create great products.

Whether you’re testing a groundbreaking medical device or refining an ecommerce website, our team of usability experts will guide you every step of the way. With decades of experience and a proven track record, we ensure that your testing process is efficient, insightful, and aligned with your goals.

At UX Firm, usability testing is one of our most popular services

Usability testing doesn’t just transform products—it transforms businesses. By delivering a product that’s intuitive, efficient, and satisfying to use, you set yourself apart in today’s competitive market. If you’re ready to see how usability testing can elevate your product development process, schedule a free consultation with UX Firm today. Together, we’ll unlock your product’s full potential.

Frequently Asked Questions

Want more practical insights? Check out these blogs

Usability Testing - A Primer

Fri, 09 Feb 2024 20:55:39 GMT

Usability Testing Getting Started Guide

When you visit a website, use an app on your phone, or software on your computer, do you have a good experience or perhaps not? If your experience is good, it’s likely because usability testing was a part of the product development process.

If your company is doing user testing for the products you work on, then you know how important it is to build user experience into an iterative design process to test, incorporate user findings, make changes to the interface, and test again.

This article is for those who may know something about usability testing but want to learn more or for those who don’t know much about it but want to get started.

What is usability testing?

Usability testing is the activity that focuses on learning how your users engage with your product or interface by watching them perform tasks that are real and meaningful to them.

Usability testing uncovers the pain points, confusing terminology, unclear navigation, and wrong assumptions about how users expect to engage with the interface (based on their “mental model” of how it should work) versus how the interface forces users to engage.

The goal of usability testing is “discovery” of your user’s experience so that you can build this learning into your product in development by strengthening what works well and reducing the pain points.

How do you conduct usability testing?

Usability testing can be conducted in person in the same space with the user and moderator or remotely from your location to your user’s location using meeting software such as Teams or Zoom. Usability testing can also be conducted through one of the software testing platforms like usertesting.com, in which users perform tasks and provide feedback without observers being present. The sessions are recorded for viewing later by stakeholders.

In this article, I will focus on in-person, do-it-yourself usability testing, which can be formative or summative.

What is formative usability testing?

Formative usability testing – also called generative testing – is testing that is done early in product development. Formative usability testing provides insights about your users’ experience at early stages of design and development and results in recommendations for changes, which allow developers and designers to course correct before code is frozen.

Whatever budget you have for usability testing, you should allocate a part of it for formative/generative testing. Why? Because if you wait to do usability testing when the product is nearly done, you miss the opportunity to make meaningful changes without investing substantially in time and cost at this point.

But if you do formative usability testing , you utilize a low-cost way to build user experience into the product’s development. Even one formative test is better than none. But repeating the formative test process along the development timeline increases your ability for continuous improvement of your users’ experience. The mantra of formative usability testing is “test early, test often.”

What are the elements of a formative usability test?

Happily, you don’t need much in terms of resources or requirements to conduct effective formative usability testing. Gone are the days when you need a formal lab like the one shown here on the left with the one-way mirror for observers to see the action taking place in the observation room, although it can be useful in some kinds of testing to have observers on site.

Usability testing can be done with just a simple set up like the one shown on the right, using a conference room or office for the sessions.

Also gone are the days when you need to schedule lots of testing sessions. Five to eight users is typically sufficient.

We know that small numbers are effective from research conducted in the early 1990s by Jakob Nielsen and others who noted that when you recruit the same subset of your users doing the same tasks, you will reach the optimal cost-benefit ratio at five users, weighing the cost of testing against the benefits gained.

At five users, you will attain discovery of 85% of the findings for the study. Rather than spend more time and money to learn the remaining 15% of findings from the study, it’s better to iterate on the design and test again.

Small studies, incremental changes, big improvement in UX.

The elements of a formative usability test are these:

A moderator/facilitator – working from an informal facilitator's guide or a more structured moderator’s guide to maintain consistency between testing sessions
A note taker – taking notes into an Excel spreadsheet or Word or Google doc to capture what users say and do
A participant – the target user selected because they have the need or goal that your interface can satisfy
A camera – not required, but good to use for session recordings if you are in a space where you can set up a camera or you’re conducting remote usability testing where the participant has a webcam

How do you recruit study participants?

It’s really important to recruit the right participants – your actual users – for usability study results to be meaningful. Getting participants is the hardest part of planning, so you want to start this task as soon as you have identified who your users are.

Small studies work well when you have real users. Grabbing “Joe down the hall” or your neighbor down the street is – dare I say – a waste of your time and budget because you can’t know if these participants’ experiences reflect your real users’ experiences. The same goes for using internal users: Even if you think internal users are OK because they are far removed from your product, they are still too knowledgeable about what your company does, the internal jargon you use, and other learned experience that should disqualify them. Unless, of course, the product is for internal users.

Recruiting participants yourself

You can recruit participants yourself via social media, your company’s website, your marketing department, your current customers, and other no-cost methods. You may not need to provide an incentive for their participation, as they may be interested enough to participate because they like to learn about new products and services that relate to their needs. However, even when these participants don’t expect an incentive, it’s nice to provide one. Consider your budget to determine what you can offer. At the lowest level, it could be a coffee mug or branded t-shirt. A better incentive is a gift card from Amazon or elsewhere in whatever amount fits the budget. And, of course, actual cash as an incentive is always an option if your budget allows it.

Recruiting participants through a recruiting company

You can also hire a recruiting company to recruit your users and pay their incentive for participating. The recruiting company will tell you what the incentive needs to be and also their costs for recruiting. Some recruiting companies recruit on a national level; others are local to your area.

If you use a testing platform, you can use the platform’s participant panel to recruit users for your study. You fill out a brief screener to target your user group, and the platform does the rest.

What happens in a formative usability study?

We’re already identified the basic setup of a usability test, which involves three people – your user, a moderator, and a notetaker. So, what do they do together?

The moderator typically does the following:

Greets the participant and makes them feel welcome
Reviews forms like an NDA (non-disclosure agreement) and video consent form
Introduces the study goals and emphasizes that “we are not testing you; you are testing the interface (or product) so that we can learn from you.”
Describes the “think aloud” protocol, which is the methodology of having the participant share their thoughts as they engage with the interface.
Asks pre-test questions to learn about the specific experience of the participant's current or prior experience with your product or competitor products and their needs and goals as they relate to your product
Describes the first task (and it can be helpful to also provide the task description in a handout)
Observes and listens and take notes as the user engages with the tasks and shares their thoughts about it. Typical think aloud responses include:
I’m not sure I’m in the right place here…
This is exactly what I thought would happen when I clicked on this link
I love how it lets me do...
Asks follow-up questions based on what was observed; probes for insights using open-ended phases, such as:
Tell me more...
I observed that you did …. What was your thought process at that time?
Avoids asking leading or biased questions, such as:
How difficult was that for you?
Did that go well?
Were you frustrated when you were trying to do that task?
If the participant asks you questions, answers with a question back to the participant, such as this exchange:
Participant – Did I do that right? Moderator – What did you expect to have happen when you did that?
Participant – I don’t know how to find what I’m looking for. Moderator – What would you want to do at this point?
Describes the next task and the next, using the same method, until you have presented all the tasks (or run out of time)
·Conducts a post-test interview to gain overall user insights on the experience, using questions, such as:
Was there anything particularly easy or difficult about doing these tasks?
If you could make one change, what would it be?
What was the best part of the experience?
Ends the session by thanking the participant and showing appreciation for the insights you have gained.

What is summative usability testing?

Summative usability testing is the opposite of formative usability testing in these ways:

Testing takes place at or near the end of product development.
The product can be evaluated against benchmarks and metrics for time on task, task completion rates, and pass/fail performance.
More sessions are required – typically 15 -20 participants.

What happens during summative usability testing?

The general approach to summative testing is similar to formative testing, engaging a target user in tasks with the help of a moderator and notes by a notetaker. Recruiting is similar, but a stipend is generally expected in summative testing, reflective of the commitment of time expected of the participant and to help assure that you reach your target for the number of sessions.

Summative testing specifics include the following:

Users engage in tasks without interruption and without thinking out loud or otherwise talking to the moderator.
The moderator’s guide is highly structured so that the same information is consistently presented to each participant; moderators often read a script to the participant.
Data is generally quantitative, with counts and completion rates noted
Formal post-test questionnaires, such as the SUS (System Usability Scale) are often used as a standard quantitative measure that can be compared against other studies of your product or the industry.
The SUS is just 10 questions, so it doesn’t take much time
It is not technology dependent, so it can apply to any interface; you might need to change the word “system” in the questions to “interface” or “application,” but that’s OK
A meta-analysis by Jeff Sauro of SUS scores across numerous industries and a variety of applications found that the average SUS score from a usability study is 68. You can use this number as a benchmark for your studies.

Which is better? Formative or summative usability testing?

The answer depends on your goals. Each type of usability test – formative or summative – can provide useful and important insights about UX (user experience).

Formative usability testing is the technique to use during product development. It’s easy to incorporate into the design cycle and, because it’s qualitative in nature, it both shows you how users engage and tells you in their own words what pleases, confuses, or frustrates them as they work through tasks. These qualitative insights form the basis of recommendations for design improvements while the product is in development.

Summative usability testing is the technique to use at the end of development, as it provides the metrics to determine if you got the usability right. In summative testing, you validate the design against measures that determine user success, user satisfaction, and user outcomes.

Whatever budget you have for usability testing, make it a priority to conduct both formative and summative usability testing. A small formative study or two early in development can greatly increase the likelihood that your product will measure up to users’ expectations in a summative study.

Want more practical insights? Check out these blogs.

Usability Testing of Medical Devices

carolbarnum@uxfirm.com (Carol Barnum) — Fri, 09 Feb 2024 20:37:13 GMT

Are you bringing a new or modified medical device to the US Market? Medical device human factors validation testing requires strict procedures from FDA and standards from ISO that have to be followed to demonstrate that your product is safe and effective to use for the intended users in their intended use environments.

What is the FDA guidance for medical device usability?

FDA requires usability testing, which is called human factors validation testing, "to demonstrate that the device can be used by the intended users without serious errors or problems, for the intended uses, and under the expected use conditions.”

This blog gives you a basic understanding of the steps and processes involved in the FDA human factors guidance, which sets forth the following requirements:

Test participants represent the intended (actual users) of the device.
All critical tasks are performed during the test.
The device user interface represents the final design.
The test conditions are sufficiently realistic to represent actual conditions of use.

Identify actual users for medical devices

To identify your actual users, you need to consider all types of users who will engage with the product, such as patients, professional and lay caregivers, doctors, and service technicians. Each type of user represents a user group. User groups are defined by their engagement with your product and their knowledge, training, or experience with devices like yours.

Here are two examples of the user groups for a human factors validation study.

Example: Cardiac monitoring device

This cardiac monitoring device is to be prescribed by a doctor for patients to use in their home over a period of weeks, in which cardiac monitoring will take place, then returned to the medical device manufacturer for cleaning, inspection, and subsequent reuse. The following three user groups are identified:

Adult patients who may or may not be experiencing cardiac-related symptoms – these users can be combined into one group, so long as it includes users with no current symptoms (to represent general population), as well as users who have experienced cardiac symptoms.
Health care professionals (HCPs), including nurses and certified medical assistants who will train the patient in how to use the device at home – these users can be combined into one group, so long as all participants have the same type of experience with similar products in setting up and monitoring electrophysiology and cardiac devices for patients.
Technical staff who will prepare the device to send out to patients and receive returned devices to prepare for reuse – this is a single user group of technicians with experience reprocessing devices for reuse. If the device manufacturer receives its devices for reprocessing in their facility, then employees of the device manufacturer can be recruited for the study; if the device is reprocessed in a clinical setting, technical staff would need to be recruited from the types of hospitals and clinics where similar devices are reprocessed.

Example: Endovascular stent graft

This device is a stent graft to be inserted by a physician for the endovascular treatment of abdominal aortic aneurysms. The single user group comprises vascular surgeons and other cardiovascular clinicians who have experience performing this type of surgery.

For each user group, a “screener” needs to be developed. The screener provides the criteria for selecting or rejecting potential participants.

Identify training requirements for medical device users

Once the user groups are identified, training requirements need to be determined. If training for any of the user groups is expected, then training should be included in the study. If training is expected but may not be available in actual use in all cases, a worst-case scenario could be identified to conduct testing without training. In this case, a separate, untrained user group may be needed for the study, along with a user group that receives training.

Generally, when training is expected, a qualified trainer (nurse, company representative) who represents the training that will be provided in real life needs to provide the training in the testing situation. For example, patients are generally trained for in-home use of a medical device by the nurse who prepares them to go home after a hospitalization. The nurses have to be trained so that they can then train the patients.

Nurses are often trained in small groups, representing the in-service training sessions they would likely receive by the device manufacturer’s representative. Physicians are typically trained individually by a company sales representative or technical support staff.

Plan for the decay period

When training is part of the process for preparing users to use the medical device, the training needs to be provided in advance of the testing session. Importantly, a “decay period” needs to be included to provide a break between the training session and the testing session.

FDA does not specify the length of the decay period, but it suggests that one hour may be acceptable in some cases. In fact, a minimum one-hour decay period is commonly used in practice, as reflecting what could be a user’s lunch break.

If more time can be scheduled between training and testing, the better the realism of the training/testing scenario will be. However, it may be challenging to recruit some clinicians, such as skilled surgeons, to schedule time for training, followed by a break of a day or a week or more, and then schedule another time for the testing session.

How we handle the decay period

In our practice, we schedule each physician in their own training session, followed by a break of one hour on the same day, and then their testing session. Participants receive an incentive for their total time for the study, which includes training, decay period, and testing.

For nurses, we schedule 3 or 4 participants for an in-service group training in the morning, then provide a minimum of a 1-hour decay period break, after which we schedule the nurses in back-to-back individual sessions in the afternoon and evening of the same day.

Fewer cancellations result when all 3 elements – training, decay, testing - are included in the same day.

Identify all critical tasks for medical device testing

For each user group, all critical tasks must be identified and validated. A critical task is defined in the FDA guideline as “a user task which, if performed incorrectly or not performed at all, would or could cause serious harm to the patient or user, where harm is defined to include compromised medical care.”

Critical risks can be evaluated through performance tasks and knowledge tasks. For performance tasks, validation studies document the number of use errors, close calls, and difficulties experienced by users. For knowledge tasks, these studies document the users’ understanding of safe use of the product beyond what can be observed in actual use. Knowledge tasks typically include questions around critical risks, such as proper storage, disposal, and reuse. A knowledge-task question might be, “How long can you safely store the device?” or “What conditions are required for safe storage?” Knowledge tasks generally require the user to find the information in the instructions for use (IFU) or in the labeling or packaging, or in recall from training or experience.

The critical tasks are derived from the URRA (Use Related Risk Analysis). The FDA identifies use-related risk based on “the combined probability, occurrence, and severity of harm.”

A URRA sets forth the systematic use of available information to identify use-related hazards and to estimate use-related risk. To do that, regulatory compliance staff at the medical device manufacturer need to establish levels of severity (typically on a scale from 1 - negligible to 5 - catastrophic) and frequency and likelihood of occurrence for each task. Those tasks identified as serious (3 or higher severity) are determined to be critical tasks.

A new FDA Draft Guidance document (December 9, 2022) provides the following table for use in presenting the critical risks identified in the URRA. The last column, the validation method, is generally indicated as the use scenario/performance task # or knowledge task #. These numbers come from the test plan included in the test protocol.

Recruit participants for medical device testing

Once the user groups, session lengths based on critical tasks, and training requirements are determined, recruiting can begin, using the screeners developed for each user group.

Recruiting participants for studies is one of the most important and challenging parts of planning for usability testing, as some participants may be difficult to locate and recruit, while others, such as specialized surgeons, may require hefty incentives to secure their commitment to participate.

If you do not have the resources to recruit the participants yourself, you will need to work with a company that specializes in recruiting participants that meet your requirements. It is important to note that participants cannot be employees of the device manufacturer, except where the user group is exclusively employees (such as service technicians used in reprocessing devices). It is also important to note that participants must be U.S. residents.

For each user group, FDA recommends a minimum of 15 completed usability testing sessions . This means that you will need to recruit and schedule additional users (generally 18 – 20 participants per user group) to allow for cancellations while still meeting the 15-person minimum.

Develop the test protocol for medical device usability testing

Once you have done the planning to identify user groups, training requirements, and the critical tasks, you can document all aspects of the human factors validation test in a test protocol. Topics typically include the following:

Device description
Study purpose
Primary test data (key event definitions, event detection methods, participant’s subjective feedback)
Test method (typically simulated use)
Test participants (recruiting, compensation, identify protection)
Test environment (location, testing space configuration, lighting and sound levels, distractions)
Test materials (device, instructions for use, labeling/packaging, and other peripheral materials needed for testing)
Test administration (moderator and notetaker)
Test agenda (scenarios of use, knowledge tasks, post-task questions, post-study interview)
Data collection method (data recording form)
Data analysis method (typically observation and subjective feedback, used to determine root cause for critical risk errors)
Test report deliverable (based on FDA’s Guidance, Table A-1, Outline of HFE/UE Report, Section 8: Details of Human Factors Validation Testing)
Appendices:
Screener(s)
Moderator’s Guide(s)

Test protocols can vary widely, and many device manufacturers have templates in place for both the test protocol and the test report. In this primer, I will focus on a few key topics in the test protocol.

Use scenarios

For the procedural/hands-on part of the study, a use scenario provides context for the user. The use scenario describes a situation that results in the participant interacting with the medical device in a natural workflow. For example, a use scenario for a clinician using a cardiac monitoring device and the associated app, might be this:

“ You are preparing to see your patient [patient name with patient information provided in a handout], who will be fitted with a cardiac monitoring device. In preparation for seeing the patient, use the app on the mobile device [provided as part of the study] to confirm the patient’s details in the mobile application. ”

Simulated use

The medical device needs to be in a production-ready state, but the device itself may need to be modified for simulated use to protect the participants from any risk while using the device. Simulated use may also be needed to provide realistic, but not actual data on a patient. Simulated use may include the use of a manikin in place of an actual patient or it may include a person recruited to take the role of the patient.

Simulated use also means that the testing environment must simulate, to the extent possible, the environment in which the device will be used. For instance, if there are noises in the environment, such as voices or the sounds of equipment in a clinical setting or music in a home setting, these can be simulated with an audio recording of the conversations and equipment sounds in a clinical setting or a music track playing music for a home settng. If lighting conditions will be low in actual use, these should be simulated. If the space in actual use is constricted, similar space restrictions need to be set up for simulated use.

Some products are best tested in a simulated use surgical suite, which some recruiting companies can provide in their testing suite. In most cases, this requirement is not needed if the test protocol documents how the simulated environment will be set up. For example, a hospital bed for a participant or a draped table with a manikin representing the patient can be used.

Key event definitions

All use errors for all critical tasks must be identified for later analysis.

FDA defines a use error as “an action or lack of action that was different from that expected by the manufacturer and caused a result that (1) was different from the result expected by the user and (2) was not caused solely by device failure and (3) did or could result in harm.”

In addition to identifying use errors, FDA wants to know about any close calls or difficulties observed and experienced by users. These are not defined as use errors, but the study needs to identify and analyze them, particularly when more than one user experiences a close call or difficulty. The test protocol needs to define a close call or difficulty, along these lines:

Close call - a case in which a participant almost commits an error, but “catches” himself or herself in time to avoid making the error. The term also describes a case in which a participant commits a use error but detects it quickly and recovers before the error becomes consequential.
Difficulty - a case in which a participant appears to struggle to perform a task. Such a struggle might be indicated by multiple attempts to perform the task, anecdotal comments about the task’s difficulty, facial expressions suggesting frustration or confusion, and higher than usual task performance times.

Other key definitions typically include:

Assistance needed – a case in which a participant requires or requests assistance to complete a task. Considered a task failure in cases where it does not represent an intended use outcome. In cases where customer support is available, requests for assistance would be noted and analyzed as to root cause, but would not be noted as a task failure.
Test artifact – an error or omission that results from the simulated-use conditions employed in testing, which are not consistent with actual use.

Moderator's guide

A moderator's guide (also called a Facilitator's Guide) is needed for each unique user group. The guide provides a script for the moderator to use in each testing session. The script includes the welcome, introduction to the study, pre-test questions about the participant’s experience/background, the scenarios/task flows being tested, post-task questions, root cause analysis for any use errors, close calls, or difficulties observed, and post-study interview questions.

The uniformity of sessions is achieved by the moderator's close adherence to the guide.

Conduct medical device usability testing sessions

The best practices for human factors validation testing puts the test protocol plan into action in individual test sessions with each user in every user group.

Testing sessions (which typically can be as short as 15 minutes or as long as 2 hours or more) provide the user experience required to determine whether the device is safe to use. The logger/notetaker uses the logging template (typically in the form of an Excel spreadsheet) to note use errors, close calls, difficulties, and any assistance needed for all critical tasks, responses to knowledge-task questions, and participants’ answers to post-task questions and post-study interview questions.

For the post-study interview, FDA suggests the following questions:

1. What did you think of the device overall?

2. Did you have any trouble using it? If so, what kind of trouble did you have?

3. Was anything confusing?

4. Please tell me about this [use error or problem observed]. What happened? How did it happen?

In our studies, we typically ask Question 2 and Question 2 above as part of post-task subjective feedback. Then we conduct root cause analysis for subjective feedback from participants on our observations of any use errors, close calls, difficulties, or incorrect answers to knowledge-task questions. Then in the post-study interview we focus on the participant’s overall experience with questions like the first one in the FDA bulleted list above, as well as these questions:

Would this device support your work? Yes/no. Explain.
Is there anything you would change?
Do you have any final thoughts you would like to share?

Write the report for FDA submission

Now that testing is complete, the job of documenting the findings to determine the root cause for all use errors, close calls, or difficulties observed for critical tasks and non-critical tasks in the task flow requires a review of the findings and root cause analysis based on the notetaker’s notes and any notes taken by the moderator and observers.

The report provides quantitative findings, counting how many participants experienced each use error, close call, and difficulty, and any incorrect responses to knowledge-task questions. For each error, close call or difficulty and each error in answering a knowledge-task question, the report provides an analysis of the cause of the error to determine the root cause. The report also provides subjective data in the form of relevant comments made by participants with respect to each critical task in which a use error, close call, or difficulty was discussed, as well as participants’ subjective feedback in the post-study interview.

How to report use-related risks and mitigations

FDA states that “the primary purpose of the analysis is to determine whether [any] part of the user interface could and should be modified to reduce or eliminate the use problem and reduce the use-related risks to acceptable levels. An essential secondary purpose of the analysis is to develop a modified design that would not cause the same problem or a new problem.”

Changes to the device design and any associated elements are called “mitigations,” as they are intended to reduce the errors noted.

However, FDA does not expect that every use error can be addressed by mitigation. If, for example, the user did not read the instructions during testing, resulting in a use error or an incorrect answer to a knowledge-task question, the moderator will want to explore the participant's actions in the post-task or post-test interview. The moderator may ask the participant to read the relevant step in the instruction. The participant may state that the instruction was clear, but they chose not to read it. The goal of the analysis of root causes of errors is to minimize the risk as far as possible. In this case, there might not be any further mitigation possible.

In other cases, the report may indicate that minor changes to the IFU or labeling, such as adding directional arrows to the interface, can aid the user's understanding of what they need to do, whether or not they read the instructions. A minor change such as this addition, it can be argued, does not require retesting.

How to decide when to re-test

Mitigations are often presented as changes needed to the instructions for use, labeling, or training. When numerous issues are identified, FDA will likely expect that further testing is needed to confirm the reduction of risk following mitigation.

In this situation, the current human factors validation test becomes a formative usability test. Retesting in a follow-up summative human factors validation study need only focus on the critical tasks that received a high number of issues in the current test.

Timeline considerations

When preparing for FDA human factors validation testing for medical devices , this introduction gives you a general idea of the process. The actual steps in planning, preparing, testing, analyzing the findings, determining root causes for critical task failures, and writing the report take time and require many more considerations than I can cover here.

How much time? On a fast track for a study with a single user group in testing sessions of one hour or less, the timeline can be completed in 10 weeks if everything is ready at the start of the planning process and the team can focus full attention on the requirements. For testing with more user groups or in longer sessions, the timeline will likely be extended.

Want more practical insights? Check out these blogs.

Remote Moderated Usability Testing

Fri, 18 Aug 2023 14:44:34 GMT

Remote moderated usability testing primer

During the pandemic, we all got used to conducting usability testing remotely, either by using Zoom or another collaborative meeting platform or using one of the remote testing platforms like UserTesting. Once the pandemic was over, many of us have continued to use remote moderated usability testing for all the benefits that derive from this approach to learning about your users’ experience in real time. If you are considering this research approach, this primer gives you tips and techniques for conducting remote moderated usability testing so that you can put this technique into action to build user experience into your development cycle.

What is remote moderated usability testing?

Remote moderated usability testing is a testing technique in which you meet your participant in real-time but not in the same place. You engage your participant in typical tasks using your product (software, website, mobile device, app) and ask them to think out loud so that you can understand what they are thinking as they do the tasks. You then interview them in a semi-structured interview format to gain insights into their experience, both the things that worked well and the things that did not.

When should you conduct remote moderated usability testing?

Anytime is a good time to conduct remote moderated usability testing, and the more often you do it during product development, the better. Designing products with user experience built in is called an “iterative design” process, which means that you respond to the results of usability testing throughout the design and development process, making the product a reflection of good user experience. Testing early at the prototype stage or focusing on a single feature is a great use of remote moderated usability testing. But it’s also effective to conduct remote moderated usability testing for current products and even competitor’s products to gain insights that support customer requirements for the next version or release of your product.

What tools can you use to conduct remote moderated usability testing?

So many tools (some of them free for first-time or limited use) support your ability to conduct remote moderated usability testing.

Using collaborative meeting software such as Zoom or Teams, you can conduct individual testing sessions with your participants from the convenience of your workspace and their home or workspace.
Using a calendar program such as Calendly, you can make your testing schedule available to your prospective participants, and they can choose the session time that works for them.
Using social media or your website for recruiting, you can invite prospective participants to participate by completing a screener in SurveyMonkey or Google forms.
Using Amazon or Tremendous, you can set up gift card payments for their participation.

How do you conduct remote moderated usability testing?

Remote moderated usability testing doesn’t take the time or cost of in-person, in-the-same-place moderated usability testing, but it still requires planning to go off well.

Here’s what you need to know to plan for success:

Start recruiting right away : Recruiting takes the most time in planning, so you want to start recruiting as your first activity. If you are recruiting yourself, you can schedule participants as they become available. More often, though, you need to schedule your complete study within a block of time. Small studies of 5 – 8 participants can be accomplished in a few days to a week or longer. So, start your planning with a plan for recruiting participants. This plan includes how you will find them, screen them for eligibility, and pay an incentive for their participation. If you don’t have the staff resources to recruit yourself but you have the budget resources, you can work with a recruiting company to do the recruiting for you. These companies typically require two – three weeks to set up, screen, and confirm the participants.
Create a test plan : Every usability test needs a test plan, but remote moderated usability testing needs some additional components in the test plan to support success. The test plan needs to include information on how you will provide the information (tasks, instructions, interface access, consent forms) to the participant. To put the participant at ease in the remote context, your introduction will likely be a bit more detailed than is needed in face-to-face in-person testing.
Prep your participants for their session : This task may require several communications (email or text, depending on the preference of the participants). You need to let them know they have been selected, confirm their session date and time, and specify the general plan for the session. If they need to use a specific browser on their computer or laptop or their smartphone, if they need to download some software for the study, if they need to review and sign a video consent and NDA (non-disclosure agreement) before testing, if they will be expected to have their camera on during the session recording, if they will be asked to take mouse control during the session, you need to provide them with all of this information. If they are uncomfortable or unfamiliar with any of these technology requests, you may want to do a dry run with them before the session to confirm that they can access the session and do what’s expected. This may be more relevant for some user populations than others.
Invite observers : Providing the opportunity for your team or relevant stakeholders is a great way to share the benefits of usability testing with everyone who has an interest or a need to understand your users’ experience. When you invite your team to your sessions, remind them of a few ground rules, such as being on mute, not having their camera on, and using Chat or another messaging app like Slack or What’s App to communicate with you and other observers during the session. Of course, when observers are in the meeting, you will want to let your participant know. I typically say something like, “We also have some observers from the company, as we are all very interested in learning about your experience today.”
Conduct a pilot session : It’s always a good idea to “test the test.” In a pilot session, which you can conduct with a team member who stands in for the participant, you want to practice working with your moderator’s guide to go over the logistics of how you will provide the tasks to your participant and manage their access to and engagement with the product interface.
Will you read each task to the participant, or put the task into the Chat for the participant to read or access via links to each task and scenario created in Google docs or something similar?
Will they need to be able to access product instructions as part of the study? If so, how will you make the instructions available?
Conduct the testing session : Get into the meeting software at least 5 minutes ahead of the scheduled start and invite your observers to do the same. You want to be ready when your participant arrives. Make small talk for the first minute or two, while you also express appreciation for their taking the time to engage with your product in the session. Confirm that they can access the product and are able to get mouse control, if that’s required. Review the video consent form and confirm that they are comfortable allowing the session to be recorded. Then start the recording. Then, using your moderator’s guide, go through the study components for the tasks and post-task questions. When you have finished, these, engage in the post-test interview, then thank the participant again, review how the incentive will be handled, and remember to stop the recording.

What is the difference between remote moderated and unmoderated testing?

Remote unmoderated usability testing is conducted without a moderator, making it asynchrononous , meaning that the test session can happen whenever the participant is available because there is no moderator needed. The deliverable is the recording, which you and your team can review afterward.

In contrast, remote moderated usability testing is synchronous , meaning that the participant and the moderator are in the testing session together at the same time. Because remote moderated testing allows for engagement of the moderator with the participant while they are performing the tasks in the study, it provides the opportunity for the moderator to probe for insights based on observations and to ask questions “off script” from the moderator’s guide, as the situation presents itself.

Remote unmoderated testing does not provide for this interaction, but the advantage to remote unmoderated testing is that it can happen at any time without the requirement of coordinating schedules with participants and moderators. Several software platforms provide the tools for conducting remote unmoderated usability testing, which include fast recruiting from the platform’s panel of potential participants and fast setup of the study specifics using the platform’s tools.

Platforms such as UserTesting and Lookback generally require a monthly (or annual) fee (although some allow for a free trial). If this approach fits your user research goals, you can use one of these platforms to get quick insights into user experience about a feature or task on one or more platforms, such as a laptop and a mobile device. If you don’t get a definitive answer to a research question, you can set up another study just as fast to dig deeper or learn from a wider pool of users. You can also get a much larger number of sessions conducted in a short period of time if you want to produce quantitative findings.

Keep in mind that there is a risk that some test participants in the panel may not be a good match for the limited number of screener questions you can ask. In some cases, you may need find that your participants are "professional testers," who may not actually match the qualifications needed for your study.

How to choose the right testing method?

It comes down to what best fits your situation.

Moderated remote usability testing has the advantage of engagement with your participants and the flexibility to ask questions in the moment. But it takes time to recruit and schedule participants, not to mention scheduling your time for these sessions.

Unmoderated remote usability testing has the advantage of speed and scale. But you get what you get: a recording of the session without the chance to ask followup questions.

One method is not better than the other. Know what you want to get out of remote usability testing and pick the method that best matches your goals. The main objective is to do usability testing so that you can understand your users and build that knowledge into better user experience.

Want more practical insights? Check out these blogs.

Microsoft Desirability Toolkit Product Reaction Words

Thu, 22 Dec 2022 21:31:34 GMT

Product reaction words

We all want to know if our users are satisfied with our product, right? And by product, we mean any interaction the user has with the interface. But how can you really tell if users are satisfied? If the product is desirable to them? If they mean what they say when they respond to structured survey questions giving you feedback on their experience?

We find out by using Microsoft’s product reaction cards.

We love the reaction cards because they give users control over the story of their experience following a usability test. We find them especially useful when combined with a quantitative feedback mechanism such as The System Usability Scale. In this blog, I'll tell you about the origin of the cards, how they work, and how we use them in our usability testing services .

Microsoft Desirability Toolkit origins

In 2002, two UX researchers at Microsoft, Joey Benedek and Trish Miner, set out to create a feedback mechanism to understand the intangible aspects of a user’s desire for a product. They wanted to know if users found products fun, enjoyable, and desirable enough to purchase.

The original “Desirability Toolkit” had two aspects:

Faces questionnaire , in which participants were asked to rate how closely their experience related to each of 6 different faces using a 7-point Likert scale to indicate their response.
Card selection exercise , in which participants were asked to choose descriptive words or phrases from a large set of product reaction cards.

Faces questionnaire

Here’s an example of a question using a face.

Look at the picture and circle the rating on the scale that most closely reflects how performing the tasks today made you feel.

The faces questionnaire proved problematic for some users, who had difficulty understanding the emotion portrayed in the face and explaining the reason for their rating. So, it was dropped from the toolkit.

Product reaction words list

The second part of the desirability toolkit - the product reaction cards - consisted of a large set of cards that participants used to select words that matched their experience.

We all know that if we ask users how they feel about a product’s desirability, they’re likely to inflate their responses to be more positive than what we observed as they engaged with the product. The reason they say nicer things than what we observed is because they want to be nice; they want to please us. It’s human nature. Microsoft kept this tendency toward a positive response in their creation of the cards, with 60% being positive words and 40% being negative words.

The Product Reaction Words list started out with 75 words but developed over several studies into the following final list of 118 words.

Using the product reaction cards in usability tests

Here’s how we use the cards in our usability tests.

For in-person studies , we have made laminated cards for each of the 118 words. We laminate them so that we can reuse them, but it’s not a requirement. You can make a new set for every study, if needed.

We spread the cards along a table so that the cards are touching but not overlapping. When the user completes the last task, we direct them to the table and provide the following instructions:

“As you can see, this table is covered in cards that have words on them representing strongly positive, strongly negative, and neutral words. There are 118 of them altogether, but they are in no particular order. What I want you to do now is walk down the table, picking up any words in any combination of positive, negative, or neutral, that reflect your experience of working with the product/interface today. There’s no set number of words to pick, but to give you an idea, pick 5 or 6 words. If you pick more, which is fine, you can tell us what your top 5 words are. When you’re finished picking your words, bring them back to me, then tell me each word you picked and what it means to you.”

After each session, we move the cards around to place them in a different order.

For remote studies , our ux research agency uses collaborative software like Zoom or Teams. When the participant finishes the last task, we show them a Word table or Excel file of the Microsoft reaction words. We then ask them to read over the rows and columns of words and select the 5 or 6 words that fit their experience. If they are comfortable using the highlighting tool in Word or Excel to make their selections, they can do this as they go along to make placeholders so that they can go back and discuss the words they picked. Or, if they want us to manage the process, we can take over mouse control and highlight the words they tell us.

The list of reaction words is alphabetical, but if you want to mix up the words on the screen so that participants see them differently each time, you can sort them differently for each session.

Customizing the reaction words

You can also customize the list of reaction words, creating your own card deck or adding some specific words to the full deck to reflect the goals of your study.

Some people go with a smaller set of words. Microsoft began with a list of 55. So, you can make the list smaller, if you like.

We use the whole set of words and we always observe that users catch on to the task quickly and really enjoy it. In just a few minutes, they have reviewed all the cards/words, they have made their selection, and they are ready to share the story of their experience with us.

But the most amazing thing is how frequently participants pick the very same word out of a deck of 118! In addition to the number of words that are the same, we also see tremendous thematic overlap in similar words chosen by participants.

Showing the product reaction word results

We typically present the reaction card results in a word cloud, which quickly shows the words chosen most often. Here's an example from a study of 23 participants.

In addition to showing the results in a word cloud, we often include a table of the positive and negative words chosen with a count of how many times each word was selected by each user. For example, the table below shows a partial list of the words presented in the word cloud, indicating the number of times it was selected and whether it was deemed a positive or negative word.graph

Reaction cards are great in small studies

You don’t need big study numbers to get great results from using the product reaction cards. Here are the cards from a single study with six participants in a day.

The same powerful consistency is clear in the results.

Reaction cards are great in iterative studies

We can also use the reaction words to establish a baseline for user satisfaction in the first study, which can then be used in iterative studies to track improvements in user satisfaction as the product goes through development or redesign. The example below is from 3 studies of a web-based application for hotel properties worldwide to implement and monitor green initiatives.

User testing of the client’s first version demonstrated that the general idea of the application was motivating; however, the product had significant problems that slowed or stopped users from achieving success. Participants’ repeated positive card choices were low with only "comprehensive," "professional" and "usable" selected twice each; but the themes of quality, appearance, ease-of-use, and motivation emerged from participants’ card choices.

The second study was of the prototype of the redesigned application. The transformation of the users’ experience was night to day. The positive card choices from 12 users now represented 82% (compared to 42% in the first study) with the most often selected card being "useful ."

With such a positive and significant measurement of change, the development team focused on the remaining issues, and a small test of the pilot version was conducted with 4 users just before launch. The pilot version results showed that all participants chose only positive words. The theme of speed now predominated and confirmed that the earlier negative issue of slow speed was now a positive feeling of fast speed—the application was fast, time-saving, and efficient .

Using product reaction cards in your UX studies

You can do this. Using the Microsoft reaction cards is easy and the results are powerful. What’s more, Microsoft has given us permission to use them with only the following disclaimer:

You don’t have to do complex diagrams to show the results. You can just count the number of times each word is chosen and present these results.

Or use a free word cloud generator to turn the word counts into a cloud. Then you can do some quick clustering of similar words to present themes emerging from the words users chose.

And, of course, use these results along with your other feedback mechanisms, such as The System Usability Scale, to compare your results from different perspectives. By triangulating your data from different sources, you strengthen your case for understanding what users showed you about their experience.

Want more practical insights? Check out these blogs.

UX Research Tools: How to Pick the Right One for your Project

Wed, 16 Nov 2022 20:15:05 GMT

What tools do UX researchers use?

UX researchers have a lot of tools to choose from when it comes to doing research to understand user experience.

So, how do you pick the right tool for your project? The choice will generally come down to time, budget, and resources. In this article, I’ll review some of the most frequently used UX tools to guide you in making the right choice for your next UX research project.

Usability testing

Usability testing is the activity of engaging your users—one at a time—in pursuit of their goals with your product and learning what works, what pleases, and what frustrates them by observing and listening to them as they think out loud.

If you could pick just one tool, the best choice is often usability testing .

Why? Because nothing can take the place of learning directly from your users while they are using your product to do real tasks matched to their goals.

Usability testing scales to fit your budget and timeline

Usability testing is a highly flexible tool. It can be scaled to fit the size of your budget and your timeline for getting results. It can be conducted in person or remotely in one-on-one sessions of variable length (15 minutes to an hour are typical session lengths).

In person testing is called moderated usability testing , whether the moderator is in the same room at the same time as the participant or in a remote location.

If usability testing is done without a moderator, it is called unmoderated usability testing, because the participant does the session without any engagement with a moderator. Instead, the participant is guided by a task list and prompted to think out loud and answer questions about their experience. The session recording is then available for the team to review after the session ends.

What you need for user testing:

For moderated testing sessions:
A plan for testing, which includes the moderator’s guide and a detailed screener for participant recruiting
A plan for recruiting (do it yourself or hire a recruiting company), using the screener to find and schedule your participants
A moderator to engage with the participant using the moderator's guide
A logger to take notes for post-test discussion and analysis
For remote in-person sessions, screen sharing/recording software like Zoom or Teams

For unmoderated testing sessions:
A testing platform like UserTesting or Loop11
Screener questions to identify your target user, which the platform will then recruit for the study
A task list for each session, typically 20 – 40 minutes long, depending on the platform chosen
Post-task and post-test questions

Expert review

As the name implies, expert review is a review of your product by a usability expert. It is also sometimes called heuristic evaluation , especially when the review is conducted using a set of heuristics—or “rules of thumb” for good usability practice. Jacob’s Nielsen’s 10 heuristics are often used and adapted for websites or other interfaces.

Expert review can be done right away

Expert review is a popular tool in the UX toolkit because it can be done almost at a moment’s notice if a UX expert is available.

The goal of an expert review is to identify violations of good UX practice. Once identified, these violations can be addressed and fixed before the product gets put in front of users.

What you need for an expert review:

One or more UX experts to inspect the interface
A set of UX guidelines or heuristics that each expert will use, formally or informally, to identify UX violations
A plan to collate the findings and prioritize the results

An expert review is a good tool to use to clean up the

interface, but it is no substitute for testing with real users.

Diary Study

How do you learn what happens with your product over time—days or weeks after its first use? A diary study is the UX tool best suited for this.

A diary study takes days or weeks

A diary study engages users in their daily activities, reporting at intervals on their use of your product in the context of their situation. A diary study is also called a "longitudinal study ," because it takes place over a period of days or weeks.

With the passage of time in a diary study, you learn how the user gets to know your product, trying out more features as they become comfortable with the basic features, and reporting on their experience.

What you need for a diary study:

A plan of activities for users to engage with your product over the period of the study
Incentives delivered at intervals in the study to encourage continued participation to the end of the study
A platform for participants to upload comments, videos, photos, and other relevant feedback

You can set up a diary study on your own, including recruiting the participants, or you can hire a UX research company to do the study for you.

You can also use a software platform like Dscout, which specializes in recruiting participants and conducting diary studies.

Ethnography

Ethnographic research is the study of people in their environment, which might be their home, their workplace, or any other place in which they will engage with your product. Ethnographic studies are also called " contextual inquiry, " because you are learning about users in the context of their own world.

Ethnography puts you where your users are

Studying people in their environment tells you what constraints they experience and what impact the environment has on their use of your product. Getting out of the artificial nature of a usability lab situation and into the real world where your users are using your product provides invaluable insights into their experience.

What you need for ethnography:

A plan for the site visit to include a script of information and questions that can be modified as the situation dictates.
Two researchers, especially if you are conducting the study in someone’s home. One researcher will take notes and one will facilitate the session.
A video camera (or phone camera) if permission is received to record the session or take photographs.

Budget, Time, and Other Considerations

If your budget is small, start with a small usability test. Five users is enough to get you started and to learn a great deal about your product.

If your timeline is short, do an expert review and then plan for a usability test in the next phase of development or budget.

If you have enough budget to use several research tools, consider this approach:

At any stage in development—the earlier the better—conduct one or more small usability tests, building improvements into the product as you iterate product design and testing throughout the development cycle.
If your product is in an early stage of development, consider a card sorting study to understand users’ expectations for grouping items into categories.
If your product is close to launch or already launched and you don’t have any UX research, conduct an expert review to identify issues to address in the next release of the product.
If your product is launched (or in pre-launch), conduct a diary study to learn your users’ experience over time.
If you want to get responses from a large number of respondents to generate quantitative findings, consider a survey, which you can deliver to respondents electronically via a link or by using a survey platform such as Qualtrics or SurveyMonkey.
If your product is in the requirements-gathering stages, conduct focus groups to get users’ perceptions of the product and their interest in it.
If you have an ecommerce website, conduct usability testing to focus on calls to action (CTAs), the shopping cart experience, and users’ reasons for abandoning the site at any point.
If you have a medical device that you want to bring to market, conduct small formative usability testing studies during product development, iterate on the design, then conduct human factors validation testing to provide the research required for FDA clearance for 510(k) and De Novo pre-market submissions

Want more practical insights? Check out these blogs.

Creating Personas Without Data

Tue, 26 Jul 2022 18:31:09 GMT

What is a persona?

A persona is a fictional representation of your user. This representation is also called an "archetype," meaning that it represents the shared characteristics, attitudes, goals, and desires of a specific group. For each unique user group, you need a persona. How many personas should you create? Enough, but not so many that they overwhelm you. We recommend 3 - 7 as a workable number.

Done right, personas "personalize" data to bring each user type to life. When a persona tells a story of their experience, the developer or designer gains empathy for this user, which can result in better designed products and a more satisfying user experience.

The First Rule of creating personas for UX is “use real data”

To be useful, personas need to be based on real data. Who could disagree with that? We certainly don’t.

Data can be collected on your users through a number of ways including:

Contextual inquiry - interviewing or observing your target users in their own environment, which could include work, home, or social spaces
Internal customer-facing resources - interviewing representatives from tech support, customer service, training, and the sales team
Interviewing customers - setting up sample customer interviews to learn the customer's experience with the product or interface, their pain points, their goals and desires
Surveys - sending surveys to current and/or prospective customers to learn their goals and pain points with your product and/or competitors' products
Web analytics - mining what you can learn from visitors to your website by studying keyword searches, pages visited, session duration, and other useful data

How to create personas when you don't have data

We know you should start with data to build realistic personas. But what can you do when you don't have data? For instance, you might be in a startup developing something never seen before, or you might be part of a team developing a new product without competitors. In other words, there is no existing data on your users.

Still, you want to build some personas to find out who your users are by testing your assumptions.

We found ourselves in just this situation with a project in which we were going to recruit participants for formative usability studies to inform product design, but we had no way to know who the target users would be.

Create fake personas as a starting point

To begin to build personas without real data, we took opinions from everyone on the project about who they thought the users would be. Then, we used these opinions to create fake personas.

As would be the method with data-driven personas, we gave our personas a name, an age, a job title, a photograph, and a key quote that reflected their lifestyle or aspirations. We described their pain points and goals, all faked, BUT all with an eye toward thinking how each persona would want to use the product.

With the characteristics of these fake personas at hand, we then created a detailed screener to find participants for our studies. Our first round of research entailed recruiting four participants from each of seven personas to engage with prototypes of the product in focus groups.

For the focus groups, we created workbooks in which the participants scored individual ratings to reflect on their

Interest in the product
Likelihood to purchase it
Identification of who (themselves or others) they thought were the target audience
Likelihood to recommend the product to others

Then we discussed each of these points with the group to gain insights in support of the participants' ratings.

In our analysis of the workbook responses, we reviewed overall ratings and responses and categorized these by persona.

The faces of our seven user persona examples are shown in the image group below.

Turn fake personas into real personas

Based on our analysis of the focus group responses and discussion with our seven representative personas, it was easy to see who was “in” and who was “out.”

From our original seven personas, we quickly narrowed the field to four subgroups of users who expressed interest in the product and were motivated to purchase it. Now we could create four personas based on real data. These personas have become the basis for additional recruiting and research.

Although we got to our personas from the unusual position of starting out with no data, we ended up with personas that we know represent our real users because we have learned who they are, as well as who they are not.

Want more practical insights? Check out these blogs.

What is a User Experience Company

Wed, 23 Feb 2022 19:55:29 GMT

A UX company builds UX into design and development

A UX (user experience) company focuses on the design and development of products that reflect good user experience. Full-service UX agencies combine the expertise of UX designers, interaction and visual designers, content creators, and UX researchers.

For each new client project, the UX agency will assign a dedicated team to meet with the client to define, design, and develop positive user experiences across all platforms and user interfaces.

Smaller UX companies will often combine roles in their staff, generally focusing their hiring on UX designers who are also expected to manage other roles, such as UX content creators and UX researchers.

Regardless of the size of the company, a UX team needs roles that encompass UX design and UX research. Job postings for UX designers often emphasize UX or UI design skills as the top skill but also seek designers who can create content and conduct UX research.

For companies that hire UX specialists and for UX job seekers, it is important to know what is generally considered to be the role of each. A look at two recent job postings by Home Depot on Indeed.com highlights the differences.

What is a UX Designer?

Here’s how Home Depot describes the job of the UX Designer

The User Experience Designer is responsible for designing intuitive, quality products through collaboration with end users and modern software development teams. User Experience Designers work directly with end users to collect and analyze quantitative and qualitative data through formal research. This information is used to inform product-related hypotheses, recommendations, and ultimately user stories in the product backlog.

20% – Planning & Analysis:
Communicates with key user and operational groups to understand business needs
Elicits ideas and assists team members with gaining shared understanding, and aligns the product team and stakeholders
Presents design concepts, user needs, and design rationale to multidisciplinary teams, as well as to key stakeholders and business partners
80% – Delivery & Execution:
Analyzes user experience data and provides recommendations to address business needs
Documents, reviews and ensures that all quality and change control standards are met
Collaborates and pairs with other product team members (UX, engineering, and product management) to design solutions
Creates designs such that information is easy to find, consume, and understand
Communicates ideas and designs to end user and product team members
Participates in and contributes to learning activities around modern software design and development core practices (communities of practice)

What is a UX Researcher?

Here’s how Home Depot describes the job of the UX Researcher

You are a passionate User Researcher, with the required training and experience conducting rigorous research across retail channels. You are adept at field- and lab-based human studies. You have worked in a research field and have examples of projects that have benefited from your results. You drive innovation by championing actionable user insights. If this sounds like you, we are looking for a resourceful, creative User Researcher to join our team, focusing on our innovative customer facing digital experiences. The User Researcher will collaborate with UX designers, product and engineering partners in conducting user research. Through deeply understanding our customers, you will identify user needs, enhance product usability, and provide our team avenues of innovation to delight our customers. Our researchers own their projects from conception and execution to analysis and presentation. To be successful in this role, you will need the ability to engage with customers, to moderate studies neutrally, and to communicate clear and actionable research findings. You will need to effectively manage ambiguity, leading large groups of designers, product managers and engineering partners toward a common goal. You will need to be a storyteller, to give life to customer data.

45% – Conduct unbiased interviews and usability sessions with customers, analyze resulting data, and create actionable deliverables so that the design, product and engineering partners are able to move forward with a clear path, knowing that they are solving customer needs. Present those findings in a succinct and engaging way to all of the project team, including designers, content strategists, and product managers and engineering partners.
45% – Create screeners to recruit target users for feedback sessions or usability studies, maintain a database/record of past and prospective participants and study design. Create moderator guide, design survey, and use online tools to create and launch research studies.
10% – Work with design, product and engineering partners to develop strategic research plans. Transform product and UX stakeholder needs into research objectives. Given project constraints, match objectives with the appropriate scope (generative or tactical) and methodologies.

What’s the key difference in UX design and UX research roles?

Although the UX designer is responsible for designing intuitive products based on knowledge of user experience, the role of the UX designer is to analyze the user research data provided by the UX researcher. The role of the UX researcher is to produce the UX research and to work with the design team to turn research objectives into recommendations that be implemented by the UX designers. In other words, these two roles are partners in creating good UX products.

What is good UX?

Good UX is achieved when users feel satisfied with their interaction with the product or interface because they can achieve their goal with ease. To sum it up, good UX is a user experience that is useful, usable, and desirable,

Peter Morville, a well-known information architect and UX expert, created a graphic to present the qualities of good UX, which he calls the user experience honeycomb . The facets in the honeycomb include both behavioral measures and the intangibles of “valuable,” “desirable,” and “credible” that users determine through their use of the product. Applying the qualities presented in the honeycomb creates awareness for everyone on the UX team to focus on good UX.

How do you know when a product or interface has good UX?

How do you know when you have achieved good UX? Your customers will tell you!

For an e-commerce website, evidence of good UX can be found in sales, indicated by a good conversion rate from browse to buy. For other types of interfaces, good UX is achieved when users post positive reviews or when calls to customer service are reduced, or training requirements are reduced or eliminated.

But indicators of good user design may not tell the whole story.

For instance, getting good conversion from browse-to-buy is not enough to assure good UX. Customers who make a purchase may push through a bad user experience with enough motivation to purchase the product, but without much satisfaction from the experience. If the experience is bad, they will often post to social media to complain about it. And they may not come back to your site, especially when there is competition that does a better job of supporting good UX. And then there are those who visited your website but left without making a purchase. Who didn’t convert and why?

Research to discover your user's experience, especially while the product is in design, provides the means to build user experience into the design throughout the development cycle.

Every change that improves user experience strengthens user satisfaction, thereby strengthening customer relations and brand loyalty. When people have a good user experience, they might tell a few friends. When they don’t have a good user experience, they are much more likely to tell a lot of their friends.

“A dissatisfied customer will tell between 9-15 people about their experience. Around 13% of dissatisfied customers tell more than 20 people. Happy customers who get their issue resolved tell about 4-6 people about their experience.” – White House Office of Consumer Affairs

And with social media at users’ fingertips, the number of friends who hear about bad user experience goes up exponentially.

How do you get good UX?

The first step in getting good UX is hiring the right people. But even hiring the best UX or UI designers doesn’t mean that they will know how users will respond to a design.

That’s where UX research comes in.

What is UX research?

UX research is a set of tools that support developers' and designers' understanding of their user's experience.

UX researchers are the people who focus on user experience from the perspective of learning from users. The most often used tool in the UX researcher's toolkit is usability testing. In one-on-one sessions with real users, UX researchers learn how users engage with the product, what’s easy, what’s difficult, what’s satisfying, and what’s frustrating. If usability testing is done early in product design and development, user experience can be built into the product as it is being designed. This process is called iterative design and the result is user-designed products.

Although usability testing is the best way to learn from users, a UX researcher has other tools in their toolkit, including:

Site visits to learn about the user’s environment and barriers or challenges that need to be addressed in product design
Customer journey mapping to “map” the user’s experience at all touch points to uncover gaps and pain points in the customer’s experience
Card sorting to understand the user’s mental model and expectations for the structure of the content on a website or software product
Diary studies to gain insights into user experience over time in an immersive context
Focus groups to learn user sentiment and attitudes regarding the company or product
Expert reviews (conducted by UX experts) to inspect an interface to uncover violations of good UX practice

The point is that many options are available to understand user experience and to make products or interfaces with good UX built in.

What is a UX consultant?

If you find that you are missing the expertise provided by a UX researcher on your team and you don’t have budget to hire a full-time UX researcher, you can seek out the services of a UX consultant. A search for UX consultant or usability consultant will produce lots of hits.

In adding a UX consultant to your design team, you add the team member whose focus is on good user experience with research based on observing and learning from your users as they identify problems and pain points that need to be eliminated.

The result? Good user experience. And good user experience is good business.

Want more practical insights? Check out these blogs.

User Insights Inform the Design of a New Customer Portal for an Atlanta Manufacturing Company

Wed, 24 Mar 2021 15:47:23 GMT

How to design an ecommerce portal that meets customer needs

Here's a story of how an Atlanta-based manufacturing company built its portal to meet customer needs.

The leadership team had a long list of features they were considering for the portal. But they didn't know what their customers would need and want.

Because the company was starting from scratch in the design of the customer portal, they didn’t want to waste time and resources building features that customers didn’t need. So, they asked UX Firm to find out what their customers wanted in a portal.

Remote moderated usability testing provide insights

You can ask people what they want in a survey or focus group, and you might gain some insights that inform design. But in our experience, nothing beats observing users engaged with the interface and learning from them what they like, what they don’t like, what they use and what they don’t use.

We got all those user insights and more through remote moderated usability testing. These came from watching and learning from our clients’ customers as they used competitors’ portals.

In one-hour sessions, each user went to a competitor’s portal where they had an account and showed us how they use the portal. We learned what they like about the portal, what they use and don’t use, and what they would change to make their experience better. If a test participant had access to more than one portal, we got them to compare and contrast their experience to show us which aspects they liked better on one portal or the other.

The client learned what worked and what didn’t work in portal design

There was a clear winner for the best portal design and several that fell far short of the goal of good customer experience.

In short order – this was a small usability study – the main features and functionality of the best customer portal became clear. Likewise, the customer portals that failed to provide a satisfying user experience also became clear.

“Nothing beats observing users engaged with the interface and learning from them what they like, what they don’t like, what they use and what they don’t use.”

The customer drives the design

Based on our findings, the client was able to quickly understand what the top issues were because the UX research showed the portal design factors that mattered most to their users.

The result was a shorter development timeline to launch the website’s customer portal.

Key takeaways from testing that you can use

Just a little user research will show you what your customers value most for a customer portal.
A small remote moderated usability study lets you see how your customers navigate competitors’ portals and learn what they like and don't like.
The results speed up the development time to launch the portal by focusing on the features that provide a good customer experience.

Want more practical insights? Check out these blogs.

What Type of Moderator Are You?

Thu, 20 Feb 2020 16:26:33 GMT

CUE-10 study debates the pros and cons of moderation style in usability testing

A recent CUE study (Comparative Usability Evaluation 10), led by Danish UX researcher Rolf Molich, brought 16 UX researchers together in Boston to debate the pros and cons of moderation style. The study participants each had to record three moderated usability tests of the budget European airline, Ryanair.com, and then view others’ recorded sessions and analyze what makes for an effective moderation style.

If the moderator spectrum has “talkative” on one end and “quiet” on the other end, where does your moderation style fall?

Talkative moderators engage more frequently in a conversational style with participants, asking more questions and probing for more insights during the session.

Quiet moderators are likely to be less chatty and take less time with meeting and greeting. They still cover all the required preliminaries, but they do so in a more reserved way. Their engagement with the participant can be viewed as an active listener.

Turns out, the CUE study analysis shows that we were fairly well divided on the moderator spectrum between quiet and talkative.

Are you a talkative or quiet moderator?

Which style of moderation reflects your style? The style you use to greet the participant and set the stage for usability testing very likely reflects your moderation style.

When you conduct the briefing at the start of a usability testing session, how do you get the participant comfortable with the process of thinking out loud? What do you tell the participant about your role as the moderator? If you sit with the participant in a face-to-face test, where do you sit in relation to the participant: next to the person, slightly behind the person, or across from the person?

Talkative moderators are likely to be chatty from the start, as they meet and greet the participant and get them comfortable.

Quiet moderators are likely to be less chatty and take less time with meeting and greeting. They still cover all the required preliminaries, but they do so in a more reserved way.

Coaches are actively engaged

Talkative moderators are likely to be using a coaching style in which they actively engage with their participants. For instance, they laugh with the participant when something funny happens, empathize when the participant experiences problems, remind the participant how valuable their reactions and thoughts are, encourage them during the process to continue sharing their feedback, and generally strive to make the participants feel like they are collaborators in the process of discovery of their experience.

Active listeners are mostly quiet

Quiet moderators are likely to be engaged in active listening . They don't say a lot during the session, but they indicate interest in the participant by using good body language (leaning in, for instance) and taking good notes to show interest. If the participant makes a comment or asks a question, they might reply with a nonverbal response (um hum) or a short verbal response (tell me more….)

"The goal of usability testing is to get viable, actionable results based on unbiased, effective moderation."

Is one moderation style better than another?

According to one study that compared three moderator styles for the same tasks, the coaching condition participants had higher completion rates for tasks than participants in the other two conditions (active listening and silence). The same was true for the number of users who gave up before completing a task. Those in the coaching condition gave up the least often.

However, participants in the coaching condition took more time to complete tasks than those in the other conditions, which isn’t surprising, given the greater engagement between the moderator and participant.

As far as satisfaction ratings, participants in the coaching condition expressed higher satisfaction with the website being tested compared to participants in the active listening condition.

And here’s another plus for the coaching style, based on a different usability study in which participants were more satisfied with the experience of being a participant when the moderator used the coaching style over the quiet style. Better results, happier participants.

Sounds like the coaching style is better all around

But wait. There can be too much of a good thing, and this is true in the coaching style of moderation. The moderator can get carried away, becoming too chatty, too conversational, too invasive in the testing session, thereby skewing the results of testing by interspersing too many questions and interruptions.

That’s where judicious silence can be effective.

Intentional silence has its place

Most of us feel awkward when silence occurs in the flow of conversation. A Dutch study showed that it takes only 4 seconds of silence for someone to become uncomfortable in a conversation.

As a moderator, are you inclined to rush in when your participant goes quiet or is slow to respond to your question or pauses in mid-sentence?

If so, you should count to 4 or maybe even 5 or 6 before saying something. Resist the temptation to finish the participant’s sentences. Give your participants time to think. While you are waiting, keep your body language positive to indicate that you are listening and engaged.

Moderation is an art, not a science

Moderators are not born; they are made. Whichever style is yours, you need to work at your craft to perfect it.

That means a novice needs to learn the rules before he or she can break them. So, what are the rules?

Best Practices when Moderating Usability Tests

Make the participant feel comfortable and valued.
Set the stage for what will be happening, why, and how, but provide flexibility.
Engage with the participant enough to show your interest in their actions and reactions.
Work with a discussion guide to follow a structured set of activities/scenarios. That generally means that you should start the scenario and then get out of the participant’s way.
Focus on your goals for the study: Is it more important to complete all of the tasks/scenarios or allow for a deep dive in any particular scenario? How much control do you need to impose regarding time on task and completion of the session?
Novices, in particular, need to follow these rules with whatever structure (a script, for instance) is required to be competent and consistent in each test session. But so do seasoned UX researchers, as consistency is an important aspect of effective usability testing.

Finally, it should be noted that some people do not make good moderators, no matter how much they practice. If you find yourself in the wrong role and you have other team members to call on, switch roles to another one that fits your personality and temperament better.

After all, we probably all can agree that the goal of usability testing is to get viable, actionable results based on unbiased effective moderation.

To learn more about being an effective moderator and all of the essentials of effective usability testing, check out my award-winning handbook Usability Testing Essentials:Ready, Set...Test! .

CUE-10 is the tenth in a series of Comparative Usability Evaluation (CUE) studies. Previous studies were conducted from 1998 to 2011. For an overview of the nine previous CUE-studies and their results see https://www.dialogdesign.dk/cue-studies/

Want more practical insights? Check out these blogs.

FDA 510(k) Clears the First ENFit Over-the-Counter Syringe

Tue, 16 Jul 2019 15:49:48 GMT

FDA supports Rx to OTC change for ENfit syringes

The Global Enteral Device Supplier Association (GEDSA) is a non-profit trade association formed to establish a voice for addressing issues that face enteral device manufacturers, suppliers, and distributors, and to help introduce international standards for healthcare tubing connectors. Comprised of manufacturers, distributors, and suppliers worldwide, GEDSA facilitates information flow about the initiative, which is designed to increase patient safety and optimal delivery of enteral feeding by reducing the risk of tubing misconnections.

GEDSA members have been actively pursuing the introduction of enteral access and delivery devices that utilize ISO 80369-3 compliant connectors commonly known as ENFit®[1]. Adoption in the United States has been slow for a number of reasons, but one, in particular, is the availability and accessibility of adequate supply of ENFit® Syringes for use outside of clinical settings by enteral feeding tube patients for feeding, hydration, and medication delivery. One of the main challenges with accessing ENFit® Syringes is their Prescription Only (Rx) designation. Unlike some historical enteral syringes that are available OTC, ENFit® Syringes require a physician’s prescription; making nutrition, hydration and medication intake a challenge for patients that live every day with an ENFit® feeding tube.

In 2017, the FDA informed enteral device manufacturers that they were open to reviewing Rx to OTC designation changes of currently 510(k)-cleared prescription-only ENFit® Syringes. For enteral device manufacturers to do this, their new FDA 510(k) submission for a designation change from Rx to OTC would have to include objective evidence obtained from controlled human factors studies with the appropriate user groups demonstrating that the instructions for using the ENFit® Syringe can be successfully followed by a patient or caregiver without the aid of clinician (21 CFR 801.5) to aid the FDA in determining substantial equivalence.

In support of this initiative, GEDSA members agreed to collaborate on the development and human factors testing of a common set of user instructions for ENFit® Syringes of varying sizes and to validate use among patients and caregivers without the supervision of a physician or pharmacist.

Study Goals for Medical Device Usability Testing

The goals of this medical device usability study were to provide objective evidence that the instructions supplied with an ENFit® Syringe can be followed by patients and caregivers in non-clinical settings without the aid of a clinician and result in the safe and effective use of the device.

Methodology for Usability Testing

RFPs went out to a number of third-party usability research firms. UX Firm was selected as the vendor. The plan called for two formative evaluations, with refinements made to the instructions after each evaluation. The summary of the results of the first and second formative studies, as well as the summative study protocol, were submitted to the FDA under the Q-Sub process for review and feedback. Feedback received from the FDA was incorporated into the test plan for the summative human factors evaluation.

The timeline for the planning, preparation, usability testing , and reporting of all three phases of research took place over the course of 18 months. Testing took place in several cities, sometimes in conjunction with industry conferences where patients and caregivers were more readily available as participants.

UX Firm planned, coordinated and conducted all three usability studies as well as drafted all associated protocols and reports.

UX Firm also engaged in regular conference calls with the GEDSA project team comprised of member representatives from several different enteral device manufacturers. UX Firm also participated in several conference calls with the FDA during the execution of this project.

Outcomes of Usability Testing

The summative human factors evaluation validated the written instruction for the ENFit® Syringes to ensure safe and effective use of the
device by lay users in the home environment.

Following the completion of testing, one of GEDSA’s member companies, which led the effort, submitted its 510(k) premarket submission to the FDA in late 2018 and received FDA clearance in early 2019 for OTC use of their ENFit® Syringes. Other companies within GEDSA will follow.

Client Response about UX Firm's Usability Testing

Here's what the client had to say:

“ GEDSA selected UX Firm in a competitive bid situation to coordinate and conduct human factors formative and summative evaluations as a collaborative effort on behalf of GEDSA members. The studies assessed the use instructions for how to properly fill, verify the dose, and administer the contents of an ENFit® Syringe. We couldn’t be happier with our choice of UX Firm for this project. Their ability to coordinate with multiple medical device manufacturers on one project while providing a high level of human factors knowledge was beyond our expectations.”

Mike Cusack

GEDSA, Executive Director

Looking for more success stories from UX Firm's engagement with medical device manufacturers seeking FDA 510(k) clearance? Check out our medical device usability testing page.

[1] ENFit® is a registered trademark owned by GEDSA and used with their permission

Want more practical insights? Check out these blogs.

Medical Device Usability Testing Participants May Need Special Handling

Tue, 03 Oct 2017 20:05:27 GMT

All usability testing participants need special handling

All participants in usability testing studies require special care and handling so that they are made to feel comfortable about the important contribution they are making to your understanding of user experience. Unless the goal of the study includes participants with disabilities—which all studies should, but often don’t—it is not uncommon to screen out people with disabilities. But in medical usability testing studies, participants may have disabilities that require additional accommodations.

In this article, we are not going into the considerable literature on how to conduct usability testing for compliance with the Americans with Disabilities Act. That’s an important topic, but not our focus here.

What we’re focusing on are the perhaps unexpected situations you may face when working with participants who may have limited mobility, hearing, vision, or other conditions generally associated with their medical condition. The need to recruit real users for medical usability research studies brings with it the potential to accommodate the specific needs of your user population.

Special situations can arise with medical device usability testing participants

Depending on the type of medical device or health care research you are conducting, you may experience the following situations that will require special sensitivities to the needs of your participants.

Service animal —participant arrives with a service animal. What do you do? Ask the participant what is best for the animal. The participant may tell you not to pet the dog and not to give it any treats. Or, the situation may be the opposite. Be prepared to do what it takes to work with the participant and the service animal, which is trained to follow the instructions of its owner.
Lip reader —the participant can speak clearly (important for think-aloud protocol) but has limited hearing, even with a hearing aid, but is able to read lips. Your lips. That means you cannot be looking at your script while you are preparing the participant for each task. You need to be sure that the participant can see your lips so that they can read them. Be prepared to depart from the exact script of your test protocol so that the participant can successfully understand the tasks and questions you will ask.
Baby (or child) onboard —participant brings a baby or child to the session. This situation is not unique to medical device or health care usability studies, but it may happen more frequently, so you want to be prepared. For instance, if you are testing with caregiver participants, they may need to bring the child they are caring for. Because your pool of participants may be more limited by the specific requirements of the user population, you will want to accommodate these situations to allow your target user to participate. Be prepared to have someone available who can entertain or care for the child, if required. In some cases, the participant may need to keep the child with them while performing the tasks in the study. Make allowances as you need to so that you can work with the participant.
Impaired or diminished mobility —a participant may be wheelchair bound or may use a walker, for example. Be prepared to rearrange your testing set up so that the participant can have access to the test elements while still in a wheelchair or if required to stand, while using a walker.
Elderly or infirm —a participant may be beyond the usual age for usability studies and because of their advanced age, may experience some infirmities. Be prepared to assist the participant in any way required to support them for the duration of the study. In a health care study we conducted with participants with macular degeneration, the participants were in various stages of this age-related eye disease. In this study, we administered drops to their eyes, assisted them in getting up onto the bed of an MRI scanner, and made sure they were comfortable for the scan. Because we knew this user population would need special accommodations to participate in the study, we had an extra team member on site to help them get onto the MRI bed and assist in other ways, as needed.

Be prepared for special circumstances

The success of any usability study is in recruiting the right participants and making them feel comfortable enough to show you their experience and share their insights with you.

When it comes to participants with additional special needs because of disability, age, illness, or a host of other factors, you need to be prepared to go the extra mile to ensure that the participants have a good experience and that you get good results.

Many interesting aspects derive from working with participants in medical and health studies. The most common one is their great attitude in wanting to participate in these types of research studies to support their community and help advance science. Since their attitude is so great, it’s our job to make the testing session fun and fruitful.

We hope that our tips get you thinking about some of the common situations you may face with this participant population. It’s not just the Boy Scouts who benefit from the motto: “Be prepared.”

Want more practical insights? Check out these blogs.

Customer Journey Maps Turn Data into Stories

Wed, 17 Feb 2016 22:37:54 GMT

What is a customer journey map?

A customer journey map is a story-telling technique that presents the story of a customer's journey from their initial point of contact through the conclusion of a transaction. A customer journey map is a visual presentation of the entry and exit points along the customer's journey, including pain points and other barriers to good user experience (UX).

A customer journey map can take many forms, but it’s likely to be an infographic that plots a particular customer’s journey along a timeline to show how the customer progresses from a desire for a solution to a problem to decision-making about which direction to take to solve the problem, and to feelings about the decision after it’s made.

Once created, the customer journey map tells the story of the opportunities and pitfalls a customer faces in taking the journey to a decision. In studying a customer journey map, an organization can clearly see the customer’s journey, which provides insights into what works well and what needs to be done better.

These insights drive design decisions to improve the customer's journey.

How do you get data for a customer journey map?

An effective, persuasive customer journey map must be based on real data about real users. How do you access this data?

Begin with data you may already have. What do you already know about your customers? There are many potential sources that you can tap into, such as:

Analytics—what do you see about user behaviors on your website?
Customer service—what issues are they reporting, what problems are they attempting to solve, what frustrations do they experience because they cannot solve problems for customers?
Sales and marketing—what does the sales/marketing staff say is their biggest challenge? Most frequent request? Reason for lost sales?
Social media chatter—what are your customers sharing with others about their experiences with your company?

What other types of data gathering do you need to add to what you already have? Some options include:

Surveys of existing customers to gather responses to specific questions about their experience.
Interviews of existing and/or prospective customers to understand their process of making a decision about your product or service. These could be your current customers, prospective customers, or customers of competitor services.
Usability studies of the inquiry-to-buying journey to discover what is important to them in making a buying decision.
Ethnography in the form of site visits to learn the environment in which users are making the inquiry-to-buying process.
Focus groups to learn what prospective or current customers consider most important in their decision-making process and the results of their decision in terms of their satisfaction level.
Ecommerce usability studies focusing on user interaction.

How many journey maps do you need?

Just as you don’t have a monolithic customer, you won’t want to create a single, monolithic customer journey map. You will most likely need to create several journey maps, based on personas of your core customers. A working number to get you started is three customer groups.

If you don’t have personas for your core customers, you can and should use the data you’ve collected to create personas for your journey map. It’s possible to have more than one customer journey on a map so long as the display doesn’t get too complicated to follow. If you have several customer personas, you will probably want to create separate journey maps for each one.

At this point in the process of mapping the customer experience and grouping the elements, it’s an excellent time to assign emotions to the findings. Are customers happy at some points, frustrated at others? Does their anticipation turn to disappointment somewhere along the path? If so, be sure to capture these emotional changes by adding sticky notes for these critical points. A subset of the journey map can be an empathy map , in which you capture the emotional changes a customer experiences on the journey.

Using your research findings to mine for great, representative customer quotes, add these at specific junctures along the journey to bring the customer’s story to life.

Be sure to identify gaps, pain points, and barriers, as these shortfalls will be important for the team to address with the goal of improving the customer’s journey.

What does a customer journey map look like?

A customer journey map can be nothing more than the physical representation of the findings, such as a whiteboard full of sticky notes that the team uses to discuss problems, map the journey, and propose solutions.

Or you can use a simple flow diagram to show the customer’s story along the path of the journey.

If you have visual or graphic design skills within your team, you can create a colorful infographic that can be published as a poster to be displayed in a prominent place for the team to review regularly.

There are many ways to present the information graphically. A quick interneet search of customer journey mapping infographics will produce lots of hits.

The common theme among so much variation is that an effective journey map shows and tells the customer’s experience in a way that brings it to life. And, as an added bonus, the effort of creating the journey map brings your collaborative, cross-functional team together to learn what it’s like for a customer to go from desire to outcome.

Seeing the results of data collection about the customer’s journey typically produces strong buy-in to fix whatever may be broken.

How often should you update your journey maps?

As with any living document, your journey maps will need new life breathed into them to keep them not only alive but kicking. The timetable to update your personas and journey maps will depend largely on your plan for implementing changes based on what you have identified needs to improve.

Once solutions are in place—even the first few—gather more data and update the journey map to indicate improvements in customer experience, while still identifying gaps and opportunities.

A good customer journey map is only as good as the data that was used to produce it. It can’t convincingly tell the customer’s story if the data isn’t there to support it.

So, you have to start with the data or you cannot build a credible story. But data must be continually gathered to feed the evolving customer’s story of the journey.

How can UX researchers support customer journey mapping?

As a UX research company , we are in a great position to make important contributions to the evolving story of the customer’s journey. We do that by providing findings from user experience research and helping the team understand and work with these findings to create and update the map.

We can use findings from prior studies to support current customer issues. Then, we can test proposed solutions resulting from the customer journey map to confirm that the solutions make the customer experience better.

Image credits: Featured image dnadigest.org/category/workshop; emotions image Richard Tait, Winning Customer Experiences http://bit.ly/1QAFrRs

Image credits: featured image–KerriBodine.com-Intuit customer journey map; 2nd image Walter Lim, FlickrImage credits:

Featured image Customer journey experience Pinterest.com; Rheumatoid arthritis journey map visual.ly/rheumatoid-arthritis-patient-journey-map designed by HealthEd

Want more practical insights? Check out these blogs.

Gender Preferences Notable During Interviews about Home Security Cameras

Thu, 30 Apr 2015 21:55:37 GMT

Does gender matter in web usability?

When planning a usability testing study, it's common practice to include a mix of genders. But, why is this necessary?

Because men and women are different. The way they engage with a website and the kinds of information they seek can often be reflected by their gender.

You may know the book Men Are From Mars, Women Are From Venus . The title alone says a lot about gender differences. Of course, we know that men and women are different. That’s why we want to have both in our research studies, unless the product or interface is targeted at only one gender.

This difference was made very clear to us In a study we conducted on home security systems. The study was focused on uncovering top priorities, seducible moments and pain points for people actively seeking a new or upgraded home security or home automation system.

Our client had a number of topics they wanted explored. The research methodology was ethnography in the form of home visits with the household’s decision-makers to learn what they wanted, how they researched the offerings, and what their main decision points were.

UX Firm conducted 20 in-home visits with the requirement that half of the users be male and the other half female. Other criteria allowed for a mix of ages and ethnicities, incomes, household makeup (including some with pets), geographic, income, and education distribution. All had to have Internet access in their home and own a smart phone.

One of the points the client wanted us to explore with participants was their interest in and need for security cameras.

As it so happened, our first 8 visits were with women. In every case, their interest in cameras extended only so far as where they wanted them located and how many came with a package. They wanted to be able to see someone at the front and back door and sometimes inside the house. And, for several who owned pets, they had a great interest in keeping an eye on their cat or dog while they were away. When asked what type of camera she was looking for, one participant summed it up for all of the women we interviewed: “I haven’t noticed” this information.

Then we started interviewing the men. Their needs and interests were focused quite differently from the women in our study. Almost all of the men were keenly interested in the specs of the cameras, saying that they wanted the latest technology. They repeatedly said that they didn’t want to see grainy images. Several said that they wanted the camera to be able to pick up the license plate of a car parked in the driveway or at the street.

Several men commented that they didn’t like the images of the cameras on the websites of the solution providers they chose to explore, calling them “dated.” As they explored further, they couldn’t find the detailed information they wanted to know about the camera, such as the resolution, the capability to pan, tilt, and zoom for the optimal setup, and whether they had night vision or LED lights. For some, they wanted this information in plain language, not tech-speak. One user said, “I don’t know what a quad splitter is.”

Gender and UX matter

If we had interviewed only women, we would have missed the essential finding that men, in many cases, wanted different information from women to help them make informed decisions about which solution providers to pursue.

We saw that most of the websites both men and women visited did not provide the information the men sought about the capabilities of the cameras in their security packages and that the images of the cameras available did not reflect the men’s desire for the latest technology.

We learned once again how important it is to have both men and women in our studies. As the popular book title reminds us, Men Are from Mars, Women are from Venus. Vive la différence!