An Exciting Alignment!
I recently came across the latest policy statement from the Human Factors and Ergonomics Society (HFES) titled ‘Supporting the Performance of Healthcare Teams.’ In this statement, the HFES strongly urges the FDA to integrate the newly devised ‘Human Readiness Level Scale’ (HRL) maturity model into the system development process to monitor the maturity of medical device design. Additionally, the HFES recommends establishing this framework as a compulsory component in government healthcare system procurement and development.
As many are aware, I have long offered recommendations closely mirroring those in the HFES policy proposals to the FDA… Therefore, I was delighted to see the HFES joining this cause! The concept of leveraging a maturity framework to systematically monitor, enhance, and standardize FDA Human Factors Engineering (HFE) for medical devices is one I have championed for years through my research and advocacy (please see references below for additional context).
Seeking Depth in HFES Policy Recommendation
However, I was curious about the specific experiences, findings, and precedents that shaped this esteemed organization’s policy proposal. Elucidating the evidence base and origins that molded the formulation of their recommendations would strengthen the rationale substantially.
In this article, I aim to trace the research origins and foundational underpinnings that likely influenced HFES’ formulation of maturity model policy recommendations for the FDA. Illuminating the precedents underlying these recommendations provides deeper insight into their rationale while also revealing inherent limitations and overlooked opportunities for more effective policy advancements (rooted in their research origins and fostering collaboration).
Agreed: Human Factors Engineering’s Critical Role in Healthcare
The HFES’ policy statement begins by painting a stark picture of preventable harm during medical care. The document highlights how human factors can bridge the gap between design and effective use of healthcare technologies.
I could not agree more that HFE profoundly impacts safety, efficiency, and user experiences in healthcare. Countless research has emphasized the significance of a systematic approach to address the usability challenges faced by healthcare professionals and patients alike (hence FDA’s requirement).
Over a Decade Since HFE Became an FDA Requirement
The FDA took its initial steps towards addressing HFE considerations for medical devices and combination products by making it mandatory, more than a decade ago, with the release of draft guidance in 2011. The guidance, subsequently finalized in 2016, solidified HFE analysis as a mandatory requirement from the FDA.
In their policy recommendation, the HFES noted that the FDA “now requires HFE” and that, despite the requirement, many approved products still have real-world clinical usage issues causing patient harm. The organization goes on to issue a compelling call to action directed at the FDA, advocating for a more substantial commitment to HFE beyond mere compliance, and recommending the adoption of the recently developed maturity model (ANSI/HFES 400-2021, Human Readiness Level Scale) in the System Development Process, as the ideal antidote to the described scenario…
An Unsubstantiated and Arbitrary Endorsement
So far, the HFES has articulated preventable harm prevalence, HFE significance, and gaps in FDA requirements. However, they did not detail the decision-making underlying their maturity model recommendations. It is crucial to establish a strong foundation. This foundation should encompass prior research and comprehensive analyses that not only endorse but also elucidate the origins and the basis of these recommendations (e.g.: any data to support the recommendation?).
With that said, the HFES policy statement appears to be disconnected from existing and ongoing work which preceded their policy statement, on the use of a maturity approach to enhance human factors practices for the specific and unique case of medical/healthcare systems. Given HFES’ prominent scientific standing, it is expected that the institution would acknowledge substantial prior contributions in this field. Surprisingly, that has not been the case.
The prestigious HFES presented their recommendations as original and innovative ideas, yet lacked necessary depth on the rationale and historical context behind proposing a maturity framework, and specifically, the HRL model. Next, I discuss my seminal research in this context, and will also provide my informed opinion regarding the implications of HFES’ arbitrary policy recommendations.
FDA HF Validation Requirement – Prior Research
In the aftermath of the FDA’s publication of final guidance in 2016, I noticed emerging challenges and made the topic “FDA human factors validations for medical devices and combination products” the focus of my PhD research at Binghamton University. During that time, the landscape revealed a conspicuous absence of endeavors aimed at pinpointing deficiencies within the FDA’s human factors requirements.
Furthermore, there were hardly any tangible solutions or remedies proposed to address any deficiencies or concerns. Then, I rigorously investigated every facet of this nascent field, as evident in my research manuscript published in September of 2019: “A Narrative Review of FDA Human Factors Validation Requirement: The Needs of Key Stakeholders and Proposal of an Industry (Human Factors Service Providers) Maturity Assessment Tool.”
The Groundbreaking SHF 2.0 Maturity Framework
After conducting systematic research on the topic and identifying the primary challenges, I put forward and created a maturity framework tailored to the industry. This framework was carefully developed, based on research, through a mixed-methods approach that encompassed industry surveys and a panel of experts. The framework was designed to offer a structured and industry-specific solution for enhancing healthcare human factors projects, with special focus on FDA’s requirements, but aligned to other applicable standards and guidelines.
My doctoral research journey culminated with the validation of the Successful Human Factors™ (SHF) framework, published in the summer of 2020. Subsequently, with the aim of making this work readily accessible to all stakeholders, I shared a summary of the validation study and findings, and also launched this online platform, as the central resource hub for the SHF 2.0 framework. This platform offers free access to stakeholders interested in advancing the application of HFE in MedTech product development, using a validated industry-focused maturity framework.
A Cookie-Cutter for all HFE Applications?
While the HFES swiftly endorsed the generic HRL framework to address current challenges in the context of medical human factors, it’s essential to consider potential limitations. As the original investigator and advocate of this approach and line of research, having conducted in-depth research on maturity models for this case, I want to underscore the significance for my use of the term industry-focused.
A well-known limitation of maturity models is their inability to serve as a ‘one-size-fits-all’ or cookie-cutter solution — a pitfall that the HRL maturity framework unfortunately fell into. While ambitious in its pursuit of being a universal maturity framework for all possible HFE scenarios, it is one among several conceptual maturity models. During my extensive literature review, I discarded multiple existing alternatives due to their lack of suitability for the case at hand, as previously documented in my 2019 publication.
Arguing the HRL Suitability for Medical/Healthcare Systems
In the domain of HFE and healthcare, a practical maturity framework demands a comprehensive approach that goes beyond human factors, allowing for strategic alignment with overall organizational processes and performance, as well as external demands. Bearing this in mind, there are several reasons why the generic HRL model may not be suitable to support the two recommendations recently put forth by the HFES (tracking maturity of Medical Device Design and Government Healthcare Systems):
- Technological Foundation Limits Nuance: The HRL model has its foundation in the Technology Readiness Level (TRL) framework, initially developed by NASA in the 1970s to measure the readiness of hardware and software for space flight. The framework’s intrinsic technological focus, while adapted, risks oversimplifying the intricate nature of human interactions in healthcare systems. The complex nuances of user interactions may not be fully translatable or captured through the constraints of a framework and scale meant to gauge technological readiness. Varying levels of “safe use” do not serve any practical purpose within the context of healthcare.
- Potential for Complexity and Redundancy: Healthcare operates under a web of regulations and standards governing various aspects of product development and safety. Adding the generic HRL model to the existing HFE requirements in the medical device industry could lead to confusion and redundancy (the HRL ends up being yet another HFE method that overlaps with usability testing and UX research). Introducing additional HFE requirements risks complicating practices. The overarching objective should be simplification, streamlining, and standardization of best practices, not compounding complexity.
- Complex Regulatory Landscape and Potential Over-regulation: Healthcare, especially in the medical device industry, has intricate and evolving regulations. Balancing the need for compliance, safety, and efficacy with the risk of over-regulation is a critical challenge. Integrating the generic HRLs into this complex landscape may inadvertently introduce excessive regulatory measures, stifling innovation, increasing development costs, and potentially limiting patient access to cutting-edge treatments. Striking the right balance between regulation and innovation is essential for progress in healthcare.
- Resource Constraints and Innovation Hurdles: Emerging and small-scale MedTech companies, including internal Human Factors Service Providers (HFSPs), often grapple with limited resources. This constraint makes the implementation and sustenance of complex maturity assessment models, such as the HRL, impractical or burdensome. Furthermore, the rigorous evaluations associated with HRLs may inadvertently discourage healthcare innovation in general. Stricter requirements may discourage the development of groundbreaking technologies, potentially slowing progress in the industry. Balancing the need for rigorous assessment with the realities of resource constraints and the imperative for innovation is a significant challenge in this industry.
- Limited Scope and Linearity: Among the foremost constraints of the HRL model is its narrow focus, primarily centered on evaluating readiness for human utilization. The model presumes linear progression across 9 levels, but human factors maturation may be more dynamic and iterative, specially in healthcare. The human readiness perspective accounts for merely a fraction of the intricate tapestry of human factors within healthcare systems. The model fails to acknowledge the intricate interplay of diverse, complex systems integral to human factors within the healthcare domain.
- Lack of Proven Efficacy: The absence of evidence in the HFES’ policy document raises uncertainties about effectiveness of the never used HRL model in improving healthcare. HFES failed to provide concrete advantages or benefits of using the model to enhance safety and effectiveness in this domain. Even the generic efficacy of HRL for human factors remains theoretical versus empirically proven. Healthcare contexts have unique complexities that may not emerge in limited validation. Comprehensive evidence is needed to justify widespread adoption and recommendation to the FDA.
- Resistance to Change: The healthcare industry, particularly the MedTech sector and HFSPs, often exhibits strong resistance to the adoption of new standards. Established regulatory norms and standards deeply rooted in the industry’s culture can make introducing substantial changes a challenging endeavor. Overcoming this resistance and establishing the value of yet another mandatory standard is an ongoing challenge. Furthermore, introducing HRLs into established project management and development processes may face resistance, particularly if it requires significant changes in existing practices.
Safety Metrics and HRL’s 9-Level Model – Lack of Practical Utility
Furthermore, it’s important to note that regulatory evaluations typically center on assessing the safety, efficacy, and quality of a fully developed medical product. These evaluations aim to ensure that the product complies with the required standards and safeguards for both patient safety and public health. As a result, regulatory reviews do not typically involve tracking the progression of a product toward completion during its development stages (which is basically what the TRL model is about).
In this context, a completed medical device is either deemed safe or unsafe, and this determination is based on the level of associated risk. That is because safety is conventionally viewed as binary, given that it primarily concerns the absence of unacceptable risks or hazards that may lead to harm. Consequently, the use of nine distinct levels of safety lacks practical utility for the stated purpose.
Prioritizing Successful Application Over Increasing Regulatory Requirements
As part of my groundbreaking research, I looked into uncovering the root causes and consequences of failures in FDA HF validation projects. And, contrary to the assertion within the HFES policy statement, my findings contest the belief that solely increasing the HFE requirements will improve outcomes; in fact, it could exacerbate existing challenges.
As highlighted in a previous press release, “Research Urges Standardization of FDA’s Human Factors Engineering Practices Within the Medical Device Industry,” I have stressed prioritizing the successful application of HFE. My contributions have consistently emphasized optimizing the quality, standardization, and overall implementation of human factors projects for medical devices. If HFE best practices are successfully applied and continuously improved, measuring abstract levels of human readiness in this context, would be redundant (that is the goal of the HFE process!). Conversely, considering the model’s subjectivity, the bureaucratic application of the HRL model does not automatically imply robust HFE practices.
Collaborative Approach for Improvement
Although the HFES seemed to have been oblivious to any prior work on this topic, instead of advocating for the introduction of additional HFE regulations, it remains crucial to first tackle the issues already revealed. The main priority should be ensuring the standardization of best practices, which necessitates strong collaboration among key stakeholders—an area currently deficient.
In healthcare human factors, a critical challenge hindering progress is isolated stakeholders reinventing existing knowledge rather than building upon it collaboratively. To unlock the full potential of HFE in healthcare, it’s imperative to break down these barriers and encourage open cooperation among key stakeholders. This involves acknowledging the achievements of the past, and collectively working towards improving existing approaches.
Conclusion
In summary, elucidating the precedents behind any recommendation can catalyze transparent collaboration to fully harness the potential of HFE in healthcare. By highlighting the origins of research and need for cooperation, my aim is to uphold integrity and drive joint innovation.
While the HFES policy statement represents a positive step towards acknowledging HFE implementation challenges, it overlooks pivotal original research contributions that provide essential precedents and context, thus weakening their rationale. Recognizing formative research efforts is equally crucial for making effective policy recommendations.
Furthermore, it’s essential to take into account the practical constraints and limitations of the HRL model within the complex healthcare domain. Achieving seamless integration raises important concerns, which the industry-focused framework, SHF 2.0, effectively tackles by aligning with organizational processes and quality systems. This approach facilitates sustainable continuous improvement (driving excellence), while eliminating the need to reinvent established knowledge.
I remain committed to actively working with stakeholders to make progress through open collective efforts. Feel free to reach out or learn more about the framework SHF 2.0. at www.successfulhf.com/framework
References:
- Rojas, K. M. (2023, March). Validating a Project Management Maturity Framework Based on the Emerging Best Practices for Successful Human Factors Projects that Require FDA Approval -Summary of Key Findings (pre-print).
- Rojas, K. M (2020). Project Management Practices and Key Factors for Success in FDA Human Factors Validations of Medical Devices and Combination Products. Doctoral dissertation, State University of New York at Binghamton. ProQuest Dissertations Publishing. Publication No. 2020. 27963841
- Rojas, K. M. (2020). Developing an Industry-Focused Maturity Assessment Tool Based on Key Factors Critical to Quality and Success in FDA Human Factors Validation Projects – Overview. Proceedings of the International Symposium on Human Factors and Ergonomics in Health Care, 9(1), 156–169.
- Rojas, K. M., Cosler, L., & Santos, D. L. (2020). Understanding Practices and Critical Success Factors of FDA Human Factors Validation Projects–Preliminary Findings.In Proceedings of the International Symposium on Human Factors and Ergonomics in Health Care (Vol. 9, No. 1, pp. 166-175). Sage CA: Los Angeles, CA: SAGE Publications.
- Rojas, K. M., Cosler, L., & Santos, D. L. (2019).A Narrative Review of FDA Human Factors Validation Requirement: The Needs of Key Stakeholders and Proposal of an Industry (Human Factors Service Providers) Maturity Assessment Tool. In Proceedings of the International Symposium on Human Factors and Ergonomics in Health Care (Vol. 8, No. 1, pp. 222-233). Sage CA: Los Angeles, CA: SAGE Publications.
- Rojas, K. M., Sharareh, N., Cosier, L., & Santos, D. L. (2019).Considering the Dynamics of FDA Human Factors Validation Requirement: Implications of Failure and Need to Ensure Project Success-A Conceptual Framework. In Proceedings of the International Symposium on Human Factors and Ergonomics in Health Care (Vol. 8, No. 1, pp. 234-247). Sage CA: Los Angeles, CA: SAGE Publications.
