Formative vs. Summative Usability Testing for Medical Devices
If you're comparing formative vs. summative usability testing for a medical device, you're likely staring down a regulatory submission, or planning one. Both types of research are important in understanding safe use of your medical device. They serve different functions and take place at different times in the development timeline.
Here's the short version of their distinct roles: formative testing improves the design. Summative testing proves it. Everything else flows from that distinction.
What Formative Usability Testing Does
Formative studies happen while the design is still moving. You put a prototype, even a rough one, in front of representative users and watch what happens. Where do they hesitate? What do they misread? Which step do they skip?
The goal is learning, not a pass/fail verdict. A formative round with 5 to 8 participants per user group will surface most of the serious use issues, and it does so while changes are still cheap. Moving a label or rewording an instruction costs almost nothing at this stage. After design freeze, the same fix can mean re-tooling, re-validation, and months of delay.
Formative work is also iterative by nature. Test, fix, test again. Two or three small rounds beat one big one, because each round confirms whether the last fix actually worked.
🩺 Bench Note: Run your first formative study before the instructions for use are final. Teams that test the IFU alongside the device almost always rewrite it, and it's far easier to do that before the document has been through three rounds of legal review.
Why You Can't Skip Straight to Summative
Some teams try to save budget by jumping directly to validation. It rarely works. A summative study that uncovers design problems is really just an expensive formative study. Now you're redesigning anyway, and re-running the validation on top of it.
Root Cause Analysis: The Part Everyone Underestimates
When use errors or close calls occur in a summative study (and some usually do), you must analyze why they happened and what the risk implications are. A pattern of errors traced to the design itself is a serious finding. The same data collected formatively would have been routine input. Collected summatively, it can stall a submission.
| Formative | Summative (Validation) | |
|---|---|---|
| Purpose | Improve the design | Prove safe, effective use |
| Timing | Throughout development | Final design, pre-submission |
| Device state | Prototypes, drafts | Production-equivalent |
| Sample size | 5–8 per user group | 15+ per user group (FDA minimum) |
| Moderator role | Can probe and explore | Hands-off, simulated real use |
| Output | Design changes | Evidence for your submission |
When to Run Each One
A workable rhythm for most device programs looks like this: an early formative study on concepts or early prototypes, a second formative round once the design stabilizes, a third if risk analysis or major changes call for it, then one summative validation on the frozen design.
In our experience, the teams that struggle most are the ones who treated usability as a single event rather than a thread that runs through development. The teams that sail through validation are the ones who found their problems two formative rounds earlier.
⚖️ Regulatory Reality:
FDA doesn't formally "require" formative testing — only validation. But reviewers can ask how you arrived at your final design, and a documented formative history is the most convincing answer there is. Skipping formatives saves a line item and creates a gap in your story.
Getting the Study Design Right
Sample size, user group definitions, task selection, and simulated-use conditions all have to hold up to regulatory scrutiny, and they're easy to get subtly wrong. Misdefine a user group, and you may be asked to repeat the study.
UX Firm has built its reputation in medical device usability testing on exactly this: study designs that satisfy both the engineers improving the product and the reviewers evaluating the submission. Led by Dr. Carol Barnum, author of Usability Testing Essentials, the team has run formative and summative studies across injection devices, diagnostics, software as a medical device, and combination products, which is why device teams bring them in early, when formative findings can still shape the design.
If you're mapping your testing plan against FDA expectations, our breakdown of
human factors validation for FDA compliance covers what reviewers look for in the validation report itself.
Two Tests, One Through-Line
Formative testing makes the device good. Summative testing documents that it's safe. Run them in that order, with honest iteration in between, and validation becomes a confirmation rather than a gamble.
Planning a device study and not sure which type you need first? Talk to the usability experts at UX Firm. A short conversation now can save a repeated study later.
Common Questions on Formative vs. Summative Testing
What's the best way to plan formative vs. summative usability testing for a medical device?
Work backward from your use-related risk analysis. Identify critical tasks first, then schedule two to three formative rounds to refine the design against those tasks, and reserve summative validation for the frozen design. Budget formative testing early — it's the cheapest risk reduction in the entire program.
Can formative study data be included in an FDA submission?
Yes, and it should be summarized. FDA's guidance expects a description of your usability engineering process, including formative evaluations and the design changes they drove. Formative data isn't a substitute for validation, but it strengthens the narrative that your final design was shaped by evidence.
Carol Barnum
Carol brings her academic background and years of teaching and research to her work with clients to deliver the best research approaches that have proven to produce practical solutions. Carol’s many publications (6 books and more than 50 articles) have made a substantial contribution to the body of knowledge in the UX field. The 2nd edition of her award-winning handbook Usability Testing Essentials is now available.
