We have noted that poorly constructed “quality measures” often fail to induce the provision of higher quality health care, and in some cases may even lead to interventions that harm patients, waste resources, and/or contribute to clinician burnout.

We therefore endeavored to develop a system for evaluating quality measures in the hope of “retiring” poor measures and providing a guide to the creation of a newer generation of clinical quality measures which might better fulfill the goals of quality measurement. In some cases, quality should be measured to inform quality improvement (QI) activities. In other situations, people are interested in evaluating the quality of care provided to make judgments about providers or to award (or reduce) payment. Though evidence for the utility and value of pay-for-performance (P4P) is lacking, we believe both have their role.

Ultimately, however we would like to see quality of care assessed in ways that allow for improvements that produce better health care and better health for patients regarding outcomes that matter to them: primarily improved quality of life, extended quantity of life, and/or reduced expenses.

While some measures, especially during a phase of testing and development, might be appropriately used for QI activities, we believe that before a measure should be implemented in ways that include accountability (e.g., public reporting or P4P programs) however, the measure should be shown to have passed a more rigorous set of criteria.

Accordingly, members of Care that Matters[1]and editors of DynaMed Plus[2]have collaborated to produce a set of 10 criteria by which the appropriateness of quality measures should be judged. This process evolved over many months and included “field-testing” of the criteria against numerous quality measures as evaluated by many physicians.

Now that we have developed and tested the criteria, we have begun the process of systematically reviewing commonly used quality measures, and we hope to publish the results soon. (Preview: many will be deemed inappropriate.)

The criteria are presented below in an overview format and a more detailed version. We invite you to comment about them by clicking here.

Overview and Details of 10 Criteria for Health Care Quality Measure Appropriateness:

Does it matter to patients?

For an outcome measure, the outcome is important to patients (directly representative of quantity or quality of life). For a process measure, the action could lead to an outcome that is important to patients.

1. Patient-oriented outcome

Patient autonomy is preserved for decisions in which reasonable, informed patients may make different choices.

2. Autonomy preserved (shared decision-making)

Is it appropriately specified?

The population is clearly and adequately specified with appropriate exclusion criteria and assessment method.

3. Denominator specification

The outcome being measured is clearly and adequately specified, including appropriate timeframe and assessment method.

4. Numerator specification

Is there sufficient evidence that benefits outweigh harms and costs?

There is sufficient evidence that, for the action(s) expected from quality measure implementation, the desirable consequences outweigh the undesirable consequences.

5. Certainty of net benefit

Measure implementation is likely to produce net benefits that justify the resources (human, material, and financial) expended on its implementation (care provision, measurement and reporting).

7. Resource use

Does the measure assess quality, independent of significant confounding factors?

Measure implementation is unlikely to motivate a significant number of health care providers to change their patient selection, clinical decision-making behavior, or reporting in ways that improve measure performance without improving health outcomes.

8. Gaming resistance

The entity for whom the quality of care is being measured can have sufficient authority, influence or capacity to affect performance on the quality measure.

9. Locus of control

Social determinants of health of the population served do not unduly influence performance on the measure.

10. Social determinants of health

[1]Ronald Adler MD, Wayne Altman MD, FAAFP, and Alicia Agnoli MD, MPH

[2]Brian Alper MD, MSPH, FAAFP, Founder of DynaMed and Vice President of Innovations and Evidence-Based Medicine Development, EBSCO Health; Alan Drabkin MD, FAAFP, Family Medicine Section Editor, Assistant Professor Harvard Medical School; and Alan Ehrlich MD