Companies that operate with FDA oversight
are required to prove that their information systems work as promised.
This process is called "validation", and it's easier said than done.
What is Software Validation?
The FDA has issued guidelines on what qualifies as a "validated" system.
Here's the highlight reel;
these quotes capture the high-level requirements that we're up against.
Motivation
The FDA’s analysis of 3140 medical device recalls conducted between 1992 and
1998 reveals that 242 of them (7.7%) are attributable to software failures.
Legal background
Software validation is a requirement of the Quality System regulation, which
was published in the Federal Register on October 7, 1996 and took effect on
June 1, 1997. (See Title 21 Code of Federal Regulations (CFR) Part 820, and
61 Federal Register (FR) 52602, respectively.)
"Least burdensome approach"
We believe we should consider the least burdensome approach in all areas of
medical device regulation. This guidance reflects our careful review of the
relevant scientific and legal requirements and what we believe is the least
burdensome way for you to comply with those requirements. However, if you
believe that an alternative approach would be less burdensome, please contact
us so we can consider your point of view.
Electronic Records (that's us)
In addition, computer systems used to create, modify, and maintain electronic
records and to manage electronic signatures are also subject to the
validation requirements. (See 21 CFR §11.10(a).) Such computer systems must
be validated to ensure accuracy, reliability, consistent intended
performance, and the ability to discern invalid or altered records.
Objective Evidence
For purposes of this guidance, FDA considers software validation to be
“confirmation by examination and provision of objective evidence that software
specifications conform to user needs and intended uses, and that the
particular requirements implemented through software can be consistently
fulfilled.”
Design Reviews
Design review is a primary tool for managing and evaluating development
projects. For example, formal design reviews allow management to confirm
that all goals defined in the software validation plan have been achieved.
The Quality System regulation requires that at least one formal design review
be conducted during the device design process.
Requirements Specification
A documented software requirements specification provides a baseline for both
validation and verification. The software validation process cannot be
completed without an established software requirements specification
(Ref: 21 CFR 820.3(z) and (aa) and 820.30(f) and (g)).
Changes to the software
Due to the complexity of software, a seemingly small local change may have a
significant global system impact. When any change (even a small change) is
made to the software, the validation status of the software needs to be
re-established. Whenever software is changed, a validation analysis should
be conducted not just for validation of the individual change, but also to
determine the extent and impact of that change on the entire software system.
Based on this analysis, the software developer should then conduct an
appropriate level of software regression testing to show that unchanged but
vulnerable portions of the system have not been adversely affected.
Independence of Review
Validation activities should be conducted using the basic quality assurance
precept of “independence of review.” Self-validation is extremely difficult.
When possible, an independent evaluation is always better, especially for
higher risk applications. Some firms contract out for a third-party
independent verification and validation, but this solution may not always be
feasible. Another approach is to assign internal staff members that are not
involved in a particular design or its implementation, but who have
sufficient knowledge to evaluate the project and conduct the verification and
validation activities. Smaller firms may need to be creative in how tasks are
organized and assigned in order to maintain internal independence of revie
Design Specification
The software design specification should include:
Software requirements specification, including predetermined criteria for acceptance of the software;
Software risk analysis;
Development procedures and coding guidelines (or other programming procedures);
Systems documentation (e.g., a narrative or a context diagram) that describes the systems
context in which the program is intended to function, including the relationship of hardware,
software, and the physical environment;
Hardware to be used;
Parameters to be measured or recorded;
Logical structure (including control logic) and logical processing steps (e.g., algorithms);
Data structures and data flow diagrams;
Definitions of variables (control and data) and description of where they are used;
Error, alarm, and warning messages;
Supporting software (e.g., operating systems, drivers, other application software);
Communication links (links among internal modules of the software, links with the supporting
software, links with the hardware, and links with the user);
Security measures (both physical and logical security); and
Any additional constraints not identified in the above elements.
User Site Testing
Testing at the user site is an essential part of software validation. The
Quality System regulation requires installation and inspection procedures
(including testing where appropriate) as well as documentation of inspection
and testing to demonstrate proper installation. (See 21 CFR §820.170.)
Likewise, manufacturing equipment must meet specified requirements, and
automated systems must be validated for their intended use. (See 21 CFR
§820.70(g) and 21 CFR §820.70(i) respectively.)
In addition to an evaluation of the system's ability to properly perform its
intended functions, there should be an evaluation of the ability of the users
of the system to understand and correctly interface with it. Operators
should be able to perform the intended functions and respond in an
appropriate and timely manner to all alarms, warnings, and error messages.
User Requirements
A very important key to software validation is a documented user requirements
specification that defines:
the “intended use” of the software or automated equipment; and
the extent to which the device manufacturer is dependent upon that software
or equipment for production of a quality medical device.
The device manufacturer (user) needs to define the expected operating
environment including any required hardware and software configurations,
software versions, utilities, etc. The user also needs to:
document requirements for system performance, quality, error handling,
startup, shutdown, security, etc.;
identify any safety related functions or features, such as sensors, alarms,
interlocks, logical processing steps, or command sequences; and
define objective criteria for determining acceptable performance.
Takeaways
The FDA's guidance is thorough to the point of intimidation.
It calls for detailed specifications and documentation
to cover all points in the software development process.
Looking past the exhaustive recommendations,
it's important to keep focus on their motivations.
At the end of the day, the FDA is responsible for protecting people from harm.
Real people, and real harm.
That turns this from a software issue to an ethical one.
Companies that operate with FDA oversight are required to prove that their information systems work as promised.
This process is called "validation", and it's easier said than done.
What is Software Validation?
The FDA has issued guidelines on what qualifies as a "validated" system. Here's the highlight reel; these quotes capture the high-level requirements that we're up against.
Takeaways
The FDA's guidance is thorough to the point of intimidation. It calls for detailed specifications and documentation to cover all points in the software development process.
Looking past the exhaustive recommendations, it's important to keep focus on their motivations.
At the end of the day, the FDA is responsible for protecting people from harm. Real people, and real harm. That turns this from a software issue to an ethical one.