pristineliving
commented
4 years ago Reflection by Abby
I started reading about CDS systems with the Shortliffe chapter on the subject, which included discussion of those early prototypes – the Leeds system, MYCIN, and HELP. There are a number of things I found really interesting, both in what I read and how I (without intention) approached the material. To start, I liked that this chapter addressed early advancements in the field. I've had some complaints about the way the Shortliffe text is written and structured, but I think inclusion of these is really crucial in understanding, a) the computational and conceptual underpinnings, and b) the progression of CDS through time as we see other advancements in the medical informatics and healthcare sectors.
I liked being able to have a frame of reference for what CDS systems looked like 'back then' as I dove more into current research and evaluation of both existing and emerging systems. It was really interesting to sink my teeth into some of these evaluation and framework development papers, and then be able to relate either some of the taxa they identified or characteristics of a system they evaluated back to these early systems I'd already digested.
Perhaps a more minor point, we discussed MYCIN in the Knowledge Representation course with John Gennari last quarter, which was a technically-heavy approach to how MYCIN was structured that formed the basis of our understanding of rule-based ontologies. I appreciated the call back to that, because it allowed me to apply my knowledge of rule-based ontologies – their structures, strengths, weaknesses – to CDS systems for an extra layer of understanding and critical thinking. I also have a good background with Bayesian statistics, so it was interesting for similar reasons. While I don't really consider myself interested in CDS systems – at least, as far as developing and evaluating them – I am interested in some of the underlying principles that drive them, so I liked being able to read about and analyze this particular type of application.
drjaber2000
1. Introduction
1.1 Traditional Decision-making for Healthcare Practitioners Needs Improvement
Traditional decision-making by doctors includes the processes of diagnosis, problem-solving, and disease management. The doctors would also act as medical consultants and interact with patients while asking and answering questions relevant to patients' complaints. In general, the process of diagnosis is based on the deduction to what information patients expose. When the medical records were still paper-based, clinicians made decisions according to the conversation with the patient. During the patient visit in the form of a consultation session, the clinicians wrote down the brief profile of the patient, summarized the main symptoms of the patients, deal with the complaints from patient, deducted for a diagnosis, and proposed a treatment plan with a prescription for examination or medicine, or a problem-solving strategy for daily life management. After the patient visit ended, the medical records, along with the one-time examination results such as X-ray or MRI films, blood sugar measures, and hormone test results, would be kept by the patients themselves. Such a workflow may have created huge healthcare burdens before electronic health records and electronic clinical support systems deployed. We will jump into more details.
The first concern was the accessibility and extraction of patient information. For instance, it was hard to extract patient information when they were sent to the emergency department with unconsciousness. And the shortage of patient profiles in such an intense use could lead to a steep increase in cost, worsening prognosis, or death. It is always ideal for the clinicians to obtain as much useful information as possible, especially for those relevant to the trigger events of the disease. In a lot of times, current symptoms or conditions were highly associated with previous exposures. Meanwhile, the medical history and the family history of the patient will help bypass potential adverse yet implicit effect of the new prescriptions or recommend for a previously working treatment plan. However, there was no such system for the patients to store longitudinal records from different patient visits incorporating various lab measures and pathological reports. Extracting and interpreting previous paper-based medical records could be time-consuming and biased. And the patients bore the risk of losing records permanently without recovery. Later, electronic health record systems were built to store the patient information in computer-readable formats. However, a lot of hospitals still use fax as the primary artifact of record delivery. It is critical for healthcare practitioners and researchers to promote the optimization of the workflow so that the patient information could be accessible and readable with harmonization and quality control for clinicians to make an inference of disease instantly.
The second concern was the medical error that may have occurred due to the restriction of knowledge, understaffing, time constraints, or the bias of patients’ self-report. Different clinicians or doctors have different methods of consultation, educational background, and hypothesis testing mechanisms. Therefore, it was likely that the diagnosis and treatment for the same patient would vary if he or she visited a different department or clinician. What’s worse is that those errors are hard to capture and yet could potentially cause irreversible harm to the patients. A potential solution is to standardize the diagnosis procedure with certified guidelines and decision analysis algorithms to proofread, generate recommendations, notify decision-making about patient abnormality, and monitor for accidental handling errors. When exhausted and trapped in understaffing, healthcare practitioners may ignore the life-changing signals or create wrong prescriptions. In addition to the medical error caused by clinicians, there might also exist a bias from patients. They may miss critical details and comorbidities due to a lack of awareness and knowledge.
Another concern is the timeliness of patient records. For instance, a delay in delivering pathological reports for cancer patients makes a huge difference. Cancer development originates from the uncontrolled proliferation of healthy cells due to genetic mutation caused by UV exposure, and/or contact with physical or chemical carcinogens. The expansion of cancer cells could be prevented before the benign cells enter the stage of metastasis. Yet if the patient is diagnosed in late stages, the burden of cost and survival will increase dramatically. Timely diagnosis and suggested prevention are recommended for reversible disease conditions. On the other hand, intense departments like pediatrics and emergency department will urge to obtain the patient records for better decision making with less bias in diagnosis and more chance of survival.
Last yet not the least essential, traditional decision making was mostly transient, monotonically involved patient- or doctor-centered procedures. For contemporary chronic disease management, it is critical to incorporate multiple healthcare stakeholders, including clinicians, nurses, caregivers, patients, and researchers, for optimized healthcare quality. Therefore, an electronic clinical decision-making system that allows various data input in the long term is in need. With its aid, it is possible to capture the environmental, behavioral, and dietary factors, suggest lifestyle change, and track the medication pattern and history for a better-rounded patient profile with intervention in satisfactory efficacy.
1.2 Partial Solution: Computerized CDS Since 1960s
Computational capability experienced giant leaps in the 1960s. Since then, scientists had started considering building a computerized system to help clinicians extracting patient information, strategize medical problem solving, and infer healthcare modeling based on clinical observations to improve clinical outcomes. Properly implemented CDS systems are expected for clinicians to help justify the diagnosis, examine the feasibility of treatments and interventions, estimate the prognosis outcomes for potential burdens or consequences, and evaluate the health conditions of patients to be urgent or not. They will also provide more information for decision-makers including clinicians, caregivers, and patients themselves, to make better-rounded healthcare decisions. Osheroff research team formally updated and defined clinical decision support (CDS) as a process that provides clinicians, staff, patients, or other individuals with knowledge and person-specific information, intelligently filtered or presented at appropriate times, to enhance health and healthcare (Osheroff, 2007).
The classical approaches of computerized CDS lie in three directions of interest. First, CDS should help clinicians to extract patient information. The visualization tools can then demonstrate the patient data as a user-friendly and readable summary of the selected patient(s) for a faster diagnosis. Meanwhile, CDS may suggest personalized alerts, remind users for medication refills or doctor visits, or even recommend relevant intervention for decision-makers at an individual level. In such a form, CDS could be used as a tool for patient status monitoring and peripheral clinical advice generator. CDS systems could also function as an information organizer to improve the outcomes of clinical decisions.
The effort of CDS implementation had started in the 1960s, focusing on the algorithms of disease-specific diagnosis, treatment suggestion, and inpatient alerts (e.g, Leeds, MYCIN, and HELP systems, with more details in Section 2.2. Incorporated the success and lessons learned from the aforementioned system, biomedical informaticians modified the emphasis of CDS decision-making in forms of infobuttons for querying standardized patient data, branching-logic for problem-specific clinical procedures and their workflows, probabilistic systems based on belief networks for decision analysis, rule-based methods for clinical inference, and ontology systems for knowledge extraction and presentation. All of the updated approaches require high quality and large amounts of patient data.
1.3 EHR and A New Era of CDS
Since the late 1990s, electronic health record (EHR) systems have been widely implemented and provided clinicians artifacts of patient data management with cost-effectiveness, larger sample sizes, and more intact forms of patient profiles with longitudinal observations. Accompanied with appropriate quality control and coding granularity during integration from various data providers maintaining, the scope of pooled EHR, ranging from tens of thousands to millions of samples, afford insight into treatment or disease effects that may be under-reported or missed in the underpowered RCTs (Nair, 2016). EHR has granted the opportunity for researchers and healthcare practitioners to assess models driven by high-throughput patient information in both population and individual levels.
The influx of patient information empowered by EHR opens the gate of CDS with less cost, higher accuracy, and more EHR-driven implementations. EHR data has been used for surveillance of clinical disease risk factors (e.g., cardiovascular disease and its risk factors such as body mass index, blood pressure, and cholesterol levels) for population health improvement initiatives (Sidebottom, 2014). EHR could also enrich the information necessary for clinical decision making by reminding higher-risk populations for regular preventive screening (e.g., cancer screening). EHR has also been used for revealing and characterizing the disease factors among the linked open data for inference and prediction (Boytcheva, 2019). By analyzing the patient data on a large scale, we may find out more cancer biomarkers and causal factors for the optimization of screening protocols and excellent comprehension of carcinogenesis for treatment and prevention. Observational epidemiology studies have found that CVD-/lifestyle-related RFs are associated with dementia risk and targeting these modifiable RFs (e.g., diabetes, hypertension, obesity, physical inactivity) could present a viable dementia prevention strategy which prevented 35% of dementia (Barnes, 2011; Norton, 2014; Broce, 2019).
Yet the new era of CDS has encountered several challenges. First, the knowledge and information management perspectives may need to change to incorporate with such a large scale of patient data. Second, CDS developers and stakeholders may need to collaborate intensively to adopt EHR for CDS with meaningful use, efficacy, and clinical performance. Last yet not least importantly, the personalization of decision-making needs to be tightly tailored to the actual needs of patients.
2. CDS Definitions and Applications
2.1 What is Clinical Decision Support?
Much of the potential value of EHR systems comes from CDS tools – whether that be health maintenance reminders, drug-drug interaction checking, dose adjustments, or order sets (Osheroff et al. 2003; Wright et al. 2011). Greater use of these systems can address continuing safety and quality problems (Wu, Davis, and Bell, 2012). Electronic medical records (EMRs) and computerized provider order entry (CPOE) systems often incorporate CDS with the explicit intent to improve patient safety (McCoy et al. 2012).
Systems that provide CDS stretch beyond the simple retrieval of information. They communicate information with consideration for clinical context; offer situation-specific information and recommendations, without making decisions themselves; and provide relevant knowledge and analyses that enable these decisions to the relevant decision-makers. At their core, CDS systems provide
The real-world application of these principles, however, can differ greatly depending on the system. Providing the right information means that information should be evidence-based, derived from a set of recognized guidelines, or based on a national performance measure. Interventions should not be solely based on expert opinion, which can often be contentious and reduce a clinician's likelihood of following the recommendations.
As healthcare becomes more of a team approach, to the right person becomes increasingly variable and important. The right person who can take action for a patient may include a nurse, physician, physical therapist, endocrinologist, or significant other. Depending on the behaviors of the patient, CDS interventions can also change roles and be reassigned to the 'right person.' For example, a patient that is resistant to advise from their physician may have their information conveyed instead to a significant other or sibling. Similarly, the information should only be presented to individuals who can take action. An alert to adjust dosage should not, for example, be given to a nurse who would not know if the dosage had already been adjusted.
Interventions may take on a wide range of implementations, such that the right format for presentation can be vastly different even for the same information. In order to identify the 'right format', developers must first identify the underlying issues and problems this tool is trying to solve. A strong recommendation for implementers is to create an inventory of current systems and tools, such that a stakeholder in a managerial role can understand the tools available, as well as tools that may need to be developed in-house or purchased from a vendor.
As technology expands in variety and adoption, the range of channels that an intervention can be delivered does, too. Today the right channel for which intervention to move through can include EHRs, personal health records (PHRs), CPOEs, apps on smartphones, or even still on paper. The appropriate channel for delivery will largely depend on the appropriate person and the appropriate information. For a physician altering dosages, the EHR may be the right channel. Alternatively, a text message to a patient's significant other may be more appropriate for alerting them of a needed appointment.
To implement a successful intervention, the involved clinical processes must be understood and documented, such that they are not disrupted or overlapped by the intervention tool itself. This is key for ensuring interventions are delivered in the right workflow For example, in the case of a patient taking aspirin, an alert that was not designed with a full understanding of the workflow is one that only comes up after the physician has completed a full order for Coumadin, indicating risk of adverse events. A more useful alert might be one that is triggered by the physician typing Coumadin in the first data input.
These five criteria are underpinned by the understanding that diagnosis involves not deciding only what is true about a patient, but what data are needed to determine what is true. This is often challenging in the face of management decisions and understanding what data must be acquired to reach an informed decision, without becoming excessive or overwhelming. Thus, development and broad adoption of CDS technology have become a broad priority, with the inclusion of CDS systems a key feature of the federal government's 'meaningful use' incentives. Computer-based CDS, in particular, has been adopted with increasing urgency as a response to the meaningful use pressure, goals of personalized healthcare, and increasing challenges with information management.
There are three basic varieties of CDS systems:
There is some consideration that 'simple DS systems' also include knowledge resources that distill medical literature and facilitate the selection of content relevant to a clinical situation. This second category of CDS systems more encompasses the computer-based approaches considered to be 'classic' CDS tools, the origins of which date back to the early 1960s and 1970's work with probabilistic reasoning and artificial intelligence. Although the systems designed in this period are for the most part no longer in use, they demonstrate clear principles for automated decision making that have continued to inspire modern systems.
2.2 Early Trailblazers
Artificial intelligence research in the 1970s applied to clinical settings largely focused on custom-tailored assessments or advice based on patient-specific data. They followed simple logic (e.g. algorithms), decision theory and cost-benefit analysis, or probabilistic approaches. Many of these underlying principles carry forward today, with systems that suggest differential diagnoses, explanations for a patient's symptomology, or interpretations and summaries of patient records as relevant to specific contexts. Limitations in scientific capabilities, data availability, and logistics prevented widespread adoption at the time, but in discussing three key examples we may see how early paradigms still exist today.
2.2.1 Leeds Abdominal Pain System
The Leeds abdominal pain system, developed at the eponymous university in the 1970s, was a computer-based decision aid that used Bayesian probability theory to diagnose patients with one of seven possible explanations for acute abdominal pain. System developers emphasized the importance of deriving conditional probabilities from high-quality data gathered by collecting information from thousands of patients. The system used sensitivity, specificity, and disease-prevalence data from these patients to digest signs, symptoms, and test results, thus calculating probabilities for each of the seven diagnoses.
The system relied on assumptions of conditional independence of the findings and mutual exclusivity and exhaustiveness of the seven diagnoses. The formulation also assumed each patient would have only one of the conditions and selected the most likely for presentation on the basis of a physician's recorded observations. Although it was never fully implemented into an emergency department role, results would have been made available within five minutes of the necessary data forms being completed.
The system's high sensitivity remains among its most notable results. Program diagnoses were correct in 91.8% of cases, where health care professionals landed between 65% and 80%. Appendicitis was one of the diagnoses and a condition often misdiagnosed, missed, or delayed. The program never failed to make a correct diagnosis of a patient suffering from appendicitis. The Leeds abdominal pain system found wide-spread use following its successful testing and with the introduction of personal computers into the healthcare delivery system.
2.2.2 MYCIN
MYCIN presents a different approach to computer-based decision support. Based on the belief that straight-forward algorithms were inadequate for clinical problems where underlying knowledge was poorly understood or often contentious, developers instead put forward a rule-based consultation system. MYCIN combined diagnoses with appropriate management of patients who have infections – i.e. antibiotic prescription and associated warnings if any.
Interacting rules represented knowledge about organisms that might be causing infections and the antibiotics that might be used to treat it. Knowledge of infectious diseases was presented as production rules – conditional statements that relate observations to associated inferences. Conclusions drawn from one of these rules may be considered as inputs to other rules when used in reasoning, and often was to represent relationships and logic. The rule structure formed a coherent explanation of reasoning that could be displayed in an English translation in response to a user's questions. With the rule-based structure, external developers could modify the system to their needs without laborious reprogramming or restructuring, simply by modifying or adding additional rules.
MYCIN was never implemented into clinical settings, and should thus be viewed through the lens of early exploration, rather than strict methodology. MYCIN demonstrated the diversity in ways of capturing and applying knowledge to medical problems, paving the way for future research and development as it exists today.
2.2.3 HELP
One of the first demonstrations of the importance of integrating CDS capabilities into EHR systems, HELP was an integrated hospital information system at LDS Hospital in Salt Lake City with a heavy basis in underlying information technology. HELP generated automated alerts when abnormalities in a patient record were noted and had an immense impact on the development of the CDS field. The applications and methodologies that underpinned HELP spanned the full range of biomedical informatics, and later added a monitoring program for storing decision logic in the conventional medical record system. There, decision logic could be viewed as rules that relate to values of data and actions that healthcare professions might be reminded to take.
HELP served as a mechanism for the event-driven generation of specialized warnings, alerts, and reports. A modern extension of this idea is clinical reminders - alerts where the triggering event is usually time, whether the age of a patient or elapsed time since an event, coupled with other conditions. Following HELP's implementation in Salt Lake City, clinical reminders were popularized in Indianapolis in 1976.
2.3 Modern Taxonomy
We can see the lasting effects of these early trailblazers on the underlying principles that guide modern and emerging CDS systems. Most systems now typically achieve their results by using some combination of:
As greater use of DS systems perpetuates through the healthcare system, research has begun to pivot towards understanding and classifying the broad range of tools both currently available and constantly emerging. Wu, Davis, and Bell (2012) compared clinical decisions to those made by military commanders and business managers: based on assessing new information, choosing courses of action in the face of major consequences, under time pressure, and with incomplete information. As such, they have proposed seven high-level DS features from non-healthcare literature that can be applied to understanding what CDS tools are:
Wright et al. (2007; 2011) also developed a detailed taxonomy, which was subsequently used to understand the breakdown and proportion of CDS usage in the healthcare system. This classification system is based on the idea that CDS systems are 'front-end tools' whose manifestations are reliant on 'back-end capabilities'. As for front-end tools, CDS systems are types of interventions available to end-users that use clinical knowledge bases and application logic. How they present information depends on the back-end capability of the EHR system infrastructure.
To deploy an alert for DDI checking, for example, the EHR system must support a trigger that fires when a new medication is ordered, be able to access medication being ordered, be able to access the patient's current medication, be capable of mathematical calculations, and be able to display an alert with actionable choices to the end-user.
According to the 2007 proposed taxonomy, there are five axes of CDS classification:
A subsequent analysis of these taxa in 2011 showed that the proportion of available CDS tools in each category ranged significantly – from 28.3% to 96.2%. 15% of all the types surveyed were found to be present in all systems, including default doses/pick lists, medication order sentences, condition- or procedure-specific order sets, DDI checking, and health maintenance reminders. It was also noted that no single system surveyed was capable of all surveyed types.
3. Experience: Successes and Failures of CPOE
One paper published in JAMA (Journal of the American Medical Association) presented 22 medication errors produced by CPOE (Koppel et al., 2005). The paper classified the errors into two big categories: information errors induced by the failure of data and system integration, and shortcoming in the human-computer interface to fit the clinical workflow.
One example of information errors induced by the failure of data and system integration is misguided dosage display. While the dosages presented by CPOE were taken by the users as being minimally effective or commonly used, they were, in fact, a reflection of the dosages that pharmacies have in stock. These should not be used as guidelines on the dosage for prescription, but users often depend on these suggestions to evaluate the range of doses to give and gave lower-than-usual doses of medications. This problem is a mismatch on understanding between the users and the developers, where the users assumed that CPOE was supposed to help guide their administration decisions, but the developers thought differently. It should be resolved by forming a consensus between the two parties. One way may be to educate users on what is the information behind the dosage display or to decide on a more intuitive way of the information displayed.
An example of shortcomings in the human-computer interface to fit the clinical workflow is selecting the wrong patient. The display on the screen makes it easy to fall into one of the pitfalls and select the wrong patient for order entry. The pitfalls include small font size, the proximity between names and drugs, patient name not shown on all screens, differences across the screens that incapacitate consistent interpretation, patient names sorted by alphabet rather than caring teams or rooms. Some of these problems exist due to a lack of investigation from a users’ perspective and insufficient knowledge of the clinical workflow. Whereas in some electronic systems, display of names is best achieved by sorting in alphabetical order, in a clinical setting, it is simply not the case. The unit to be used in a clinical setting is the caring teams or rooms. This makes much more sense in a clinical setting and it is easier to search and click on the correct patient name.
Other errors of information errors induced by the failure of data and system integration include failing to cancel a drug prescription after administering a new one, failing to cancel a medication when it is linked to a procedure delayed or canceled, orders given immediately or given-as-needed are not recorded and result in duplication of medication, gaps in antibiotic therapy due to the uncoordinated renewal process, delay in warning of allergy, etc. And other errors regarding shortcoming in the human-computer interface to fit the clinical workflow include selecting the wrong medication due to difficulty to view patients’ medication in one screen, giving medication to the wrong patient due to failure to log off by the previous user, information loss during system’s maintenance time, etc.
Some advantages of CPOE from multiple references were also listed in the paper. These advantages were based on comparison with paper-based systems. The advantages can be categorized in the following areas. 1) Reduction of errors related to handwriting. This includes the elimination of problems associated with reading illegible handwriting, errors related to similar drug names, mistakes on the number of zeros written. It is safe to say that the error regarding handwriting can be mostly eradicated. 2) Improved efficiency. The orders can be sent to pharmacies directly, which can also reduce mistakes made during the steps of sending the orders. 3) Availability of data use. It is easier to integrate with EHR and CDS. It is also easier to perform data analysis since the data is stored in an electronic system.
4) Improve safety on drugs. It can show warnings about drug choices, drug dosage, drug-drug interactions, drug-disease interactions, drug age (geriatric).
References
Campbell, R. (2013). The Five Rights of Clinical Decision Support: CDS Tools Helpful for Meeting Meaningful Use. Journal of AHIMA. 84(10): 42-47
Connelly, T. P., & Korvek, S. J. (2019). Computer Provider Order Entry (CPOE). In StatPearls [Internet]. StatPearls Publishing.
Kawamoto, K., Hongsermeier, T., Wright, A., Lewis, J., Bell, D. S., & Middleton, B. (2013). Key principles for a national clinical decision support knowledge sharing framework: Synthesis of insights from leading subject matter experts. Journal of the American Medical Informatics Association, 20(1), 199–207. https://doi.org/10.1136/amiajnl-2012-000887
Koppel, R., Metlay, J. P., Cohen, A., Abaluck, B., Localio, A. R., Kimmel, S. E., & Strom, B. L. (2005). Role of computerized physician order entry systems in facilitating medication errors. Jama, 293(10), 1197-1203.
McCoy, A. B., Waitman, L. R., Lewis, J. B., Wright, J. A., Choma, D. P., Miller, R. A., & Peterson, J. F. (2012). A framework for evaluating the appropriateness of clinical decision support alerts and responses. Journal of the American Medical Informatics Association, 19(3), 346–352. https://doi.org/10.1136/amiajnl-2011-000185
Osheroff, J. A., Teich, J. M., Middleton, B., Steen, E. B., Wright, A., & Detmer, D. E. (2007). A Roadmap for National Action on Clinical Decision Support. Journal of the American Medical Informatics Association, 14(2), 141–145. https://doi.org/10.1197/jamia.M2334
Shortliffe, E. H., & Cimino, J. J. (2014). Biomedical informatics: Computer applications in health care and biomedicine. London: Springer.
Wu, H. W., Davis, P. K., & Bell, D. S. (2012). Advancing clinical decision support using lessons from outside of healthcare: An interdisciplinary systematic review. BMC Medical Informatics and Decision Making, Vol. 12. https://doi.org/10.1186/1472-6947-12-90