Among all the patient safety reports that CHPSO receives from members, about 50% end up classified in the catch-all event type of “Other” rather than in a specific clinical or other meaningful event type category (e.g., healthcare-associated infection, fall, medication error, etc.). Sometimes the reports are categorized into the “Other” category because the facts of the safety event do not fit into one of the existing Agency for Healthcare Research and Quality’s (AHRQ) Common Formats specific event type categories. The broad nature of this “Other” category, diversity of events that it contains, and large size relative to the more specific event type categories renders it not particularly useful for identifying emerging patient safety issues or tracking trends over time.
To address this limitation and provide better information for CHPSO members, we are making efforts to reduce the size of the “Other” event type category and classify the reports contained therein into more descriptive event types using data gleaned elsewhere from those same patient safety reports. We are exploring three ways to reclassify “Other” reports into more meaningful categories:
1. Unmapped Event Type Data
Because of differences between our members’ local patient safety reporting systems’ formats (e.g., Midas, RL Solutions, RL Datix, and Verge) and AHRQ’s Common Formats for Event Reporting into which all CHPSO-reported safety events are converted, fields from our members’ systems sometimes cannot be mapped to an AHRQ common format. These unmapped fields often contain valuable information that we’ve been exploring for use in post hoc event type classification of events currently classified in the “Other” category. As a result, we’ve uncovered many event types that were originally unseen in AHRQ common format fields and the potential for a much richer classification of event.
2. Event Description
Event descriptions are probably the most important and complex data available in patient safety reports. These are the narrative descriptions of the events in free text format. Because of this, it can be a valuable resource for event classification. We often observe reports that contain multiple themes (for example a report can include a patient fall and a medication error). However, existing implementation of the AHRQ Common Formats for Event Reporting does not support the classification of reports into multiple event types.
We’re building machine-learning natural language processing models to solve this multi-classification challenge. These models use terms and term combinations from existing reports that are classified into specific event types to identify “Other” event reports that should be similarly classified into each specific event type. So far, our results are promising, not just being able to classify “Other” reports into existing AHRQ event type categories, but also for classifying all safety reports into newly identified categories and allowing for multiple categorizations of all events. Our clinical analyst has read through hundreds of patient safety reports to identify important event categories that are not in part of the existing AHRQ taxonomy, and we’ve confirmed that our models can successfully discern underlying event types from the event’s description.
3. Benchmarking with Other Patient Safety Organizations
We reached out to one of our strategic partners, the Pennsylvania Patient Safety Authority, to learn from their experience with event classification taxonomy. Unlike California, patient safety reports are mandated by law to be submitted in the state of Pennsylvania. Their taxonomy includes 10 major categories and 98 subcategories. Their percentage of “Other/Miscellaneous” events is about 8% compared to our 50%. Enlightened by their taxonomy structure, we are considering a significant revision of our event type classification taxonomy, including an expansion our numbers of event categories by including both major categories and subcategories. Expect to hear more from us about this effort in the future.
Together the combination of these three approaches is expected to dramatically decrease the percentage of patient safety events classified as “Other” in the CHPSO database, both historically and moving forward. This richer classification of patient safety events is expected to provide more sensitive and actionable information to CHPSO members, so they derive more meaning out of participation.
