Chapter 9 Summary
Performance Assessment for Healthcare Organizations
Peter Arthur Woodbridge - University of Nebraska Medical Center and Mid-West Mountain Veterans Engineering Resource Center and Nebraska-Western Iowa Healthcare System
George Oscar Allen - Nebraska-Western Iowa Healthcare System
1. Performance Measures Rationale
“Performance measures “motivate, illuminate, and communicate” (Metzenbaum, 2006).
Performance measures motivate by enabling tangible goal-setting and factual feedback, which in turn clarifies expectations, promotes accountability, and shape behavior.
Performance measures illuminate by establishing objectivity and consistency, leading to enhanced problem solving, modeling, decision making, and progress tracking.
Performance measures communicate by increasing the visibility of goal attainment and performance gaps, thereby focusing on organizational attention, understanding, and organizational alignment.”
- Ch 9. Handbook Healthcare System Design
The chapter makes some general observations about goal setting. Specific goals are key. Individual commitment, a belief in the importance of the goal, and ‘self-efficacy’ (the belief you can achieve the goal) is needed among users to achieve the goals. Break the goal into smaller steps, and track each step’s progress.
I don’t agree with the chapter’s claim that performance is not affected by who sets the goal. From my experience, (and John Doeer’s work on OKRs) it seems ensuring individuals help set some goals themselves is important.
Beware of creating “performance measurement-driven organizational dysfunction”. People try to game the system and work to achieve the goal - while at the same time producing undesired counter results.
The other way performance measures may create organizational dysfunction is in some ways the opposite. Those who are adverse to goal setting may spend their time working to discredit the performance metrics and refuse to strive for them.
One use where performance metrics should not be compared publicly is in ranking people. Doing so means that half of the people ranked will leave with a lower sense of “self-efficacy”. This is a problem as self-efficacy is an important ingredient in being able to carry through future goals.
2. Healthcare Performance Measurement
Performance measures are tough to balance the right number and right difficulty. Whenever performance measures are introduced they should not be tied to performance until they are validated.
Balancing measures may be required to prevent gaming of the system and organizational dysfunction. These are set against performance measures as a means to curb unanticipated consequences.
Donabedian Framework of healthcare performance measures
This framework doesn’t appear very useful, so I’ll skim through it.
The framework has three components: “organizational structure influences processes and processes in turn determine outcomes”
a. Structure: physical space, support services (EHR), equipment, human resources. eg. clinical reminder to measure A1c.
b. Process: activities to provide care. Frequency of measuring A1C.
c. Outcomes: proximal outcomes closely relate to development of distal outcomes.
either “distal” - improved life expectancy, leg amputation, blindness.
or “proximal” - intermediate, A1C reduction
Horizontal and Vertical Integration
Vertical integration: “connects departments to organizational strategies”. This turns high level goals and ideas into tangible work
Horizontal integration: “connects the activities of departments to each other”. This breaks down silos.
IOM Healthcare Performance Measure Design Principles
In 2006 IOM proposed the following 9 design principles for performance measures:
accurate and credible
support multiple uses intrinsic to care
express the patient’s voice
provide multiple perspectives
2. Healthcare Performance Measurement Systems
There are many healthcare performance measurement systems. The chapter lists seven of the more common ones. These span monitoring of inpatient and outpatient disease burden and treatment, home care services, nursing home quality, medical equipment, and investigation of morbidity and mortality.
Performance Assessment in the Veterans Health Administration
As discussed in Chapter 3, the VA system started to make significant improvements to its care in the 1990s. This included implementing capitated funding, EHR systems, and performances measures.
The VA started with an “external peer review program” (EPRP) that randomly pulled patient’s charts to review them. This has since expanded to include hundreds of performance measures. In 2005 an Inpatient Evaluation Centre (IPEC) was created to track mortality, length of stay and evidence based practice.
3. Healthcare Performance Measure Barriers
a) Performance measures are often developed from clinical guidelines. However, the development process for clinical guidelines varies, and may not be evidence based. This can lead to low confidence in the performance measurements created from some clinical guidelines.
b) Collecting data about performance measures is difficult without an EHR. Even with an EHR it is still difficult because the system often does not capture performance measurement data.
c) People love to create more performance measures, and the number to track keeps growing. What is the focus? Which are truly important?
d) When organizations all begin to achieve the performance measurements, the difference in results between sites may become insignificant, and the differences identified may be blown out of proportion - leading to unfair stratification between ‘good’ and ‘bad’ groups.
e) Other industries have moved to ‘performance measure aggregation’ and the creation of dashboards with composite scores. There is no method to aggregate healthcare performance measurements. It is unclear how to incorporate catastrophic outcomes (eg death). Depending on how you aggregate scores (eg. all-or-none, 70% standard, overall percentage, indicator average, patient average) you end up with vastly different composite results.
4. Future of Healthcare Performance Measurement
The book, published in 2010, suggests that two ways to improve performance measures: “Natural language processing” and “business intelligence”.
The hope is that by using NLP, unstructured data can be used for performance measurements. This real time data can then be used to create ‘operational business intelligence’ with near real time reporting of performance measurements (instead of the current lag of days, weeks, or months).