The Food and Drug Administration (FDA) requires the application of human factors (HF) in the development of medical devices and combination products. The previous is checked through proper validation testing. The purpose of the HF validation is to demonstrate whether a device can be used by its intended users without any serious errors or hazards, or problems with the intended uses, under certain expected conditions.
Rojas et al. (2019) analyzed the often-occurring failure of HF validation projects, which could be taking a toll on key stakeholders. After the remarks made by Dr. Rojas, the FDA started sharing relevant statistics (in 2019 and 2020 at the International Symposium on Human Factors and Ergonomics in Health Care), which indicate that over *90% of HF submissions are rejected in the first attempt.
"We find our clients get pushed back most often if they did not assess all user groups, did not perform an HF validation test, or did not perform a complete risk analysis."
To delve into the main reasons why many HF validation projects fail, Dr. Rojas carried out a research study and developed a validated survey instrument specifically designed to address these questions. The survey was filled out anonymously by individuals experienced in leading HF validations that seek FDA approval. Participants could be categorized as pertaining to either manufacturing organizations, usually procurers of HF services (although they could also have internal HF departments) or agencies/consulting firms (usually providers of HF services). Through thematic analysis and descriptive statistics applied to the collected data, the main reasons for the failure were eventually unearthed. A few of the findings from this research are shared in this post. More of the findings will also be shared briefly in future posts (e.g.: some characteristics of the HF validation projects, main challenges and the key factors for the success).
So, How Often and Why?
While about 35% of survey participants claimed that their HF submissions are never rejected by the FDA, 60-65 percent of respondents said that their submissions do get rejected sometimes. The sharp contrast about what determines project failure or success among stakeholders of FDA HF validation projects, is obvious.
"Deficiencies because of misunderstanding or because of other companies’ HF Studies."
Manufacturers/sponsors of medical devices, the FDA and HFSPs, have entirely different perceptions about the success of HF validation submissions. From the agency’s perspective, less than 6% of submissions are successful in the first attempt, which means more than 90% of submissions fail. To learn more about this, read “Considering the Dynamics of FDA Human Factors Validation Requirement: Implications of Failure and Need to Ensure Project Success – A Conceptual Framework.”
In the survey, participants were also prompted about why they fail, and some respondents seemed to want to minimize the issues (“Just misunderstandings”) or avoided specifying the exact problems due to which they get rejected by the FDA. However, those who admitted the problems, specified the following as the main reasons:
*As per FDA’s data
Summary of results of Dr. Rojas’ pioneering research on the reasons for failure in FDA HF validation projects (2019)
Behind Schedule and Other PM Issues
On another hand, running behind schedule seems to be an issue among those surveyed. Astonishingly, 100% of manufacturing organizations indicated they usually run behind schedule when it comes to the delivery of their medical device HF validations. Also, 70% of agency/consultancy firms experience such issues. In line with this, it is important to note that most of the organizations reported assigning two to three projects per employee. Similarly, participants said that, at a minimum, five to ten hours weekly are spent per project on PM tasks. Accordingly, PM tasks may add up to at least 30 hours per week, prompting the question: what happens to the HFE work? Considering that amount of work combined with impromptu PM, scheduling and quality issues that lead to project failure should not be surprising.
“Aggressive and unrealistic client timelines when HF is not prioritized from the start of product development, starting HF too late, trying to "fix" a bad design with labeling/training.”
More Formative Studies & Sponsor Awareness/Commitment
Another important reason for failure is probably the fact that formative studies are not always part of HF validation projects, and they should be “phase I” of the HF validation. It can be said that HF service providers (HFSPs) face limitations from sponsors in carrying out these studies.
Sponsors seem to lack awareness and commitment regarding the HF component in the development of quality medical devices and subsequent impact on successful pre-market applications. In general, perhaps due to a need to better understand and grasp the significance and implications of the HF requirement when it comes to successful approval/clearance to market (this was discussed in detail before).
The high-level phases of an HF validation project should include formative studies prior to the HF validation. If sponsors do not give HFSPs the opportunity to plan the HF work early in product design and development, there is a high risk that the HF validation will not be successful, regardless of the efforts of the HFSP. With that, this research went to the next question: What are the Main Challenges of FDA HF Validation Projects?
To find out more, read the full paper: Understanding Practices and Critical Success Factors of FDA Human Factors Validation Projects–Preliminary Findings. In Proceedings of the International Symposium on Human Factors and Ergonomics in Health Care (Vol. 9, No. 1, pp. 166-175). Sage CA: Los Angeles, CA: SAGE Publications.
