We have noted that poorly constructed “quality measures” often fail to induce the provision of higher quality health care, and in some cases may even lead to interventions that harm patients, waste resources, and/or contribute to clinician burnout.

We therefore endeavored to develop a system for evaluating quality measures in the hope of “retiring” poor measures and providing a guide to the creation of a newer generation of clinical quality measures which might better fulfill the goals of quality measurement.  In some cases, quality should be measured to inform quality improvement (QI) activities.  In other situations, people are interested in evaluating the quality of care provided to make judgments about providers or to award (or reduce) payment. Though evidence for the utility and value of pay-for-performance (P4P) is lacking, we believe both have their role.

Ultimately, however we would like to see quality of care assessed in ways that allow for improvements that produce better health care and better health for patients regarding outcomes that matter to them: primarily improved quality of life, extended quantity of life, and/or reduced expenses.

While some measures, especially during a phase of testing and development, might be appropriately used for QI activities, we believe that before a measure should be implemented in ways that include accountability (e.g., public reporting or P4P programs) however, the measure should be shown to have passed a more rigorous set of criteria. 

Accordingly, members of Care that Matters[1]and editors of DynaMed Plus[2]have collaborated to produce a set of 10 criteria by which the appropriateness of quality measures should be judged. This process evolved over many months and included “field-testing” of the criteria against numerous quality measures as evaluated by many physicians.

Now that we have developed and tested the criteria, we have begun the process of systematically reviewing commonly used quality measures, and we hope to publish the results soon. (Preview: many will be deemed inappropriate.)

The criteria are presented below in an overview format and a more detailed version.  We invite you to comment about them by clicking here.

 

Overview of Criteria for Health Care Quality Measure Appropriateness:

Does it matter to patients?

1.    Patient-oriented outcome

2.    Autonomy preserved (shared decision-making)

Is it appropriately specified?

3.    Denominator specification

4.    Numerator specification

Is there sufficient evidence that benefits outweigh harms and costs?

5.    Certainty of net benefit

6.    Measure implementation improves outcomes

7.    Resource use

Does the measure assess quality, independent of significant confounding factors?

8.    Gaming resistance

9.    Locus of control

10. Social determinants of health

 

Detailed version of Criteria for Health Care Quality Measure Appropriateness:

1.     Patient-Important Outcome

For an outcome measure, the outcome is important to patients (directly representative of quantity or quality of life). For a process measure, the action could lead to an outcome that is important to patients.

2.     Patient Autonomy (Shared Decision Making Preserved)

Patient autonomy is preserved for decisions in which reasonable, informed patients may make different choices.

3.     Denominator Specification

The population is clearly and adequately specified with appropriate exclusion criteria and assessment method.

4.     Numerator Specification

The outcome being measured is clearly and adequately specified, including appropriate timeframe and assessment method.

5.     Certainty of Net Benefit for Quality Care

There is sufficient evidence that, for the action(s) expected from quality measure implementation, the desirable consequences outweigh the undesirable consequences.

6.     Likelihood of Net Benefit for Measure Implementation

There is sufficient evidence that population-based implementation of this quality measure will lead to desirable consequences that outweigh the undesirable consequences.

7.     Appropriate Resource Use (Net Value Gain)

Measure implementation is likely to produce net benefits that justify the resources (human, material, and financial) expended on its implementation (care provision, measurement and reporting).

8.     Resistance to Adverse Manipulation (Gaming Resistance)

Measure implementation is unlikely to motivate a significant number of health care providers to change their patient selection, clinical decision-making behavior, or reporting in ways that improve measure performance without improving health outcomes.

9.     Empowerment of Entity Measured (Locus of Control)

The entity for whom the quality of care is being measured can have sufficient authority, influence or capacity to affect performance on the quality measure.

10.  Social Determinants of Health (vs. Quality Discrimination)

Social determinants of health of the population served do not unduly influence performance on the measure.

 

[1]Ronald Adler MD, Wayne Altman MD, FAAFP, and Alicia Agnoli MD, MPH

[2]Brian Alper MD, MSPH, FAAFP, Founder of DynaMed and Vice President of Innovations and Evidence-Based Medicine Development, EBSCO Health; Alan Drabkin MD, FAAFP, Family Medicine Section Editor, Assistant Professor Harvard Medical School; and Alan Ehrlich MD