Bryan joined Mitch Robbins on the RA/QA Today podcast to discuss “dark corners” in RA/QA - areas where RA/QA involvement is late or lacking. Bryan and Mitch discuss mergers, acquisitions, product development, change control, and other activities where involvement of qualified RA/QA professionals is critical in the early stages to ensure success.
Trust but Verify: It’s OK to Ask Questions
From due diligence for investments and M&A to internal and supplier auditing, being bold enough to ask questions may ultimately save your organization and reputation.
In the medical device and biologics industries, there is a lot of pressure to keep the ball rolling while also keeping it in bounds. Navigating financial progress must be made while remaining compliant to regulatory requirements and ensuring safety for the patients and users of your products. Such a tightrope is a challenge but is also certainly possible with the right resources and skills. This article evaluates one approach that individuals can take in any role and at any level in an organization: asking questions.
I continue to marvel with disgust at the massive ruse Theranos management deployed against patients, medical professionals, investors, business partners, regulators, and the public at large. This was one of several recent debacles that stain the reputation of a largely compassionate and honest industry. As I learn more about the Theranos debacle, my curiosity digs at the endless stream of lies perpetuated (whether intentionally or naively) by Elizabeth Holmes, her executive leadership, and top shareholders. How could they have such audacity and fearlessness in leveraging such lies to advance the company? Why did investors, potential business partners, and more media outlets not question the many claims that have now been exposed as total lies? Why were outsiders so accepting of Theranos’ inability to offer concrete evidence of any claims? In my own work, I’ve instantly identified more subtle cases of false and misleading information – why did it take so long for that to happen with Theranos? Let’s probe some examples so you understand my bewilderment and how I would’ve responded as a certified auditor and experienced due diligence professional.
First, Elizabeth Holmes claimed that the Theranos diagnostic device was deployed in medevac helicopters in Afghanistan by the Department of Defense. A second whopper was that the Theranos device did not need FDA approval to be marketed in the US. To her credit, Ms. Holmes stated that although the device did not require marketing authorization, they would seek FDA approval (actually, this supposed strategy highlights the possibility that Theranos had no regulatory affairs staff or consultants). The final fib we’ll discuss is how Theranos convinced potential investors that pharmaceutical companies were using the device in clinical trials. I understand the need for tact during discussions regarding acquisitions, mergers, or investment and the hesitation to appear incredulous. But these are perfect opportunities to probe for information simply on the grounds of genuine curiosity! If the negotiations are so fragile that asking legitimate questions will jeopardize the deal, that may highlight the suspect nature of the idea or organization for sale.
As an auditing and executive professional, some questions I would’ve asked when faced with the claims of medevac use would have been:
· What is the use and even advantage of the technology on a medevac helicopter? How does the technology in this application meet user needs?
· What feedback has the company received from combat zones on the advantages compared to other methods of testing?
· How is a precision instrument running 200 tests on two drops of blood validated to function in such a rough environment?
· What documentation is available demonstrating DoD or FDA approval for such use of the device?
· Do you have a photograph or video of the product being used in this application?
· What training was provided for use of the device?
Regarding the claim that FDA approval is not required for such a device, did anyone question such a definitive statement lacking any supporting detail? As a regulatory expert, I understand I approach this with much more knowledge than the average individual. However, did such a bold claim with such serious financial and safety implications not warrant some insistent questioning or background investigation? During due diligence, taking information at face value is a dangerous gamble. In this situation, I would’ve asked for:
· An FDA response to a 513(g) Request for Information;
· FDA pre-sub meeting minutes;
· Even informal documentation from FDA such as email correspondence; or,
· A regulatory strategy or rationale from a qualified regulatory affairs professional.
When Theranos implied the device was being used by pharmaceutical companies in clinical trials, were no follow-on questions posed by these investors? At a minimum, some additional information or verification would have been possible if this claim was true and asking for confirmation would not be out of line. If presented with papers documenting clinical performance of the device and branded with the logos of major pharmaceutical companies implying a partnership, I would ask:
· Which of the 200 tests performed by the Theranos device were being used in each study and why? Any investigational use (drug or device) should correspond with an entry on clinicaltrials.gov (hint, hint!).
· What feedback did Theranos receive from the investigators?
· Most importantly, would Theranos provide references or contacts at the pharma companies?
Theranos may very well have responded citing a cloud of secrecy and insisting confidentiality. Which in turn would prompt another question – given the supposed confidentiality, why leverage the relationships in the first place? Without a reference or some evidence of legitimacy, I would treat that information as null and possibly counterfeit. To proponents of a deal, this may seem extreme but skepticism is entirely reasonable without evidence.
Why was there seemingly such eagerness to patently accept the claims made by Holmes et al.? Why did such shallow and unsubstantiated lies prop up Theranos so long and enable them to receive billions in investments? It seems that the answer to this question is that the right questions were not asked or there was no persistence in the line of questioning. In my extensive history of auditing and performing due diligence assessments, I’ve never regretted asking questions, I’ve only regretted not. At the worst, my research and analysis were ignored. But even in those situations, the pressure test I applied to the organization was suitable to have identified any egregious violations.
Ensure you have the right team involved in auditing and assessing target companies and recipients of your investment funds; qualified quality and regulatory personnel are essential. During acquisitions and investment due diligence, don’t let the excitement blind you to reality. When auditing, hold firm on your line of questioning and don’t allow yourself to be distracted or placated. The sting of regret from poor due diligence outlasts and outweighs the excitement of the accomplishment. As stewards of the finances, safety, and well-being of so many, we must ask questions even if it’s uncomfortable. As penned by Suzanne Massie (Ronald Reagan’s adviser on Russian affairs) and stated often by President Reagan during diplomacy with the Soviet Union, “Trust, but verify.” No one should criticize you for asking legitimate questions; if verification were not necessary, due diligence and audits wouldn’t be performed in the first place.
Brosseau Consulting is available to assist your organization with due diligence, auditing, or other business matters requiring keen evaluation and analysis. Please contact Bryan here for more information and a complimentary initial consultation on how Brosseau Consulting can assist you in meeting goals and minimizing risk.
Preparing for an FDA Inspection or Unannounced Notified Body Audit
To begin, let me advise that there is no crystal ball for FDA inspections and unannounced Notified Body audits and no single tool to guarantee you a successful outcome. There are numerous variables that all contribute to the results of an FDA inspection or Notified Body audit of your organization (I will refer to both as “inspections” for simplicity). There are however measures you can take to improve your chances of a successful inspection.
Two recent FDA inspections in which I participated had very similar results on paper - each company received a 483 with one or two relatively minor observations. However, the companies were in different states of compliance - one with a robust quality management system employing best practices and the other with numerous non-compliance and quality issues resulting in multiple recalls. 483s are now issued quite frequently and your goal should be to minimize, if not eliminate, any observations. For the latter example, the results could have been much worse. However, sufficient planning and preparation was employed in advance of the inspection. And, the audit was “managed” by me and the staff in a manner to minimize the findings. The best way to prepare for an inspection is to ensure your quality system is effective, your staff is ready, and you have experienced guidance during the inspection.
QUALITY PLANNING
The first step in preparing for an inspection is to ready your quality management system. An effective quality management system (QMS) is a regulatory requirement and critical to the safety of your patients and the satisfaction of your customers. Therefore, regardless of any upcoming inspection you should ensure your QMS is effective. You likely already have a good idea of any weaknesses in your QMS. My recommendation is to establish a documented plan for improving these areas to:
a) ensure that improvement efforts for weak areas are monitored and actively managed,
b) engage staff and promote quality at all levels of the organization, and
c) demonstrate to any inspectors that you recognize the need for improvement and are acting on this need.
I have extensive experience in the effectiveness of this strategy in remedying non-compliance and mitigating inspection risks. As you work through the plan, you may identify QMS process linkages that need to be strengthened or other areas that require improvement. Feel free to modify the plan at any time (using revision control of course) to add additional requirements. You may use your existing quality system monitoring and improvement methods to track progress of the action items.
Assign action items to responsible members of your company with authority for implementing changes and hold them accountable. I recommend weekly meetings to assess the progress and ensure the plan is completed in a timely manner, particularly if substantial remediation is required. If the team is progressing quickly through action items, recognize their accomplishments and give them additional time to work on their action items by reducing the frequency of the recurring meetings. Ultimately, their responsibilities and success in your organization are also affected in the event of an unsuccessful inspection. It is easier to prevent an FDA 483 or warning letter concurrently with existing workload than it is to remedy a 483 or warning letter concurrently with existing workload. However, executive management must also recognize the efforts required and assign resources appropriately. Consider a consultant, temporary staff, or reassigning qualified staff temporarily to address quality system improvement efforts. Remember, regulators are increasingly focused on management responsibility for the quality management system and assignment of adequate resources!
MOCK INSPECTION
Organizing a mock regulatory inspection of your facility is my second recommendation in preparing for the inevitable inspection. A mock regulatory inspection should be conducted by an independent auditor with substantial regulatory audit experience. This fresh perspective and experience from actual regulatory audits is important in identifying weak areas in your organization prior to an inspection. While you should preserve the mock inspector’s independence, I do advise you notify them of the areas for improvement you’ve already recognized. This will allow them to see those areas from a new perspective while also probing other areas of your quality management system for potential weakness. The results from the mock regulatory inspection may feed into your quality plan for QMS improvement described in the previous section (again, changes to the plan must be managed with revision control). Such mock inspections are also useful in providing a general assessment of the health of your QMS. Therefore, even if you’re not expecting an FDA inspection or unannounced audit from your Notified Body, mock regulatory inspections are quite useful. Also, a mock inspection may be leveraged as an internal audit.
The mock regulatory inspection also provides your company an opportunity to iron out the logistics of hosting an FDA inspection or unannounced Notified Body audit. That is, you can arrange the appropriate resources (conference room, computer, communication methods, staff resources, etc.), assign roles and responsibilities for inspections, and document a plan to follow when the inspector arrives at your door. Particularly when you engage an independent auditor for this activity, you can evaluate your staff’s interaction with inspectors and auditors. This allows you to identify the best audit leads in your company and provide constructive feedback on their handling of the mock inspection.
IMPLEMENTING CHANGE
After identifying areas for improvement and establishing a plan, your team will begin changes to processes, procedures, forms, work instructions, and other documentation. Again, recognizing the resources required for such activities is the responsibility of executive management. For each change, your team should evaluate the feasibility, the efficiency, the complexity, and the effectiveness of each change to address the challenge. I recommend frequent and regular reminders to the team that added process steps or complexity do not necessarily improve the process. You must attain a healthy balance between documented procedural steps and verifiable employee competence and training. One recurring example where this balance is essential is for adverse event regulatory reporting (MDRs and Vigilance reports). There is no algorithm, flowchart, or other tool that can capture the detail of every adverse event and the various nuances and permutations. As such, trained and competent staff are essential as are clear and unambiguous procedures. The natural tendency to add detail to procedures and additional fields to forms is understandable but additional complexity presents opportunities for additional breakdowns. In an inspection, your staff must be able to confidently explain and defend processes requiring a combination of procedural detail and staff competence and training. Again, a mock regulatory inspection is valuable in preparing your staff for such a scenario.
The two main criteria by which your staff should judge processes and process changes are effectiveness and compliance to applicable regulations. I recommend that you make no revisions without comparing the changes to the applicable regulations. That is, employees should compare procedures to 21 CFR 820, ISO 13485:2016, Health Canada Medical Device Regulations, Australian Regulatory Guidelines for Medical Devices, Japan Pharmaceuticals and Medical Devices Act, or other regulatory requirements as applicable. One should literally open the regulations and compare them line by line to the procedures they are authoring, assessing, or revising. Consider documenting this line-by-line analysis in the change record for the revision.
To assess the effectiveness of these process changes, I recommend following the revised procedures for fictional scenarios while probing for potential breakdowns. These effectiveness verifications should be performed by the employees who will complete the actual tasks and documented in the report used to close out the quality plan. Consider also using this activity as a way of demonstrating competence for these employees. By observing or “scoring” staff on the processes, you are evaluating both the process and the employee.
SUMMARY OF RECOMMENDATIONS
This approach allows you to engage your staff, demonstrate executive management’s commitment to an effective quality management system, and let everyone experience ownership of quality. This empowers your staff, promotes quality within your organization, and establishes cohesiveness among your team. The efforts of identifying weaknesses and addressing them demonstrates to auditors and inspectors that you are committed to the continued effectiveness of your quality management system (a regulatory requirement!). When the auditor or inspector is convinced of this commitment, the outcome of the inspection will be more favorable. In summary, I leave you with these recommendations:
- Engage executive management to obtain the necessary resources to prepare for inspections
- Perform a mock regulatory inspection to identify areas of improvement
- Implement QMS improvements through a quality plan with inputs from the mock inspection
- Verify the effectiveness of the QMS improvements and the competence of your staff
If you would like to arrange a mock regulatory inspection, a quality system improvement plan, a contract internal audit, or any other quality services please contact Brosseau Consulting. Or, we can discuss other options to improve your odds in a regulatory inspection. You can contact me by email to bryan@brosseauconsult.com or by telephone at 770-855-7372.