The Consequences of Reporting on Health Care Quality

Print

With a growing public interest in addressing problems in the current health care system, a variety of health care reform proposals are underway. A common theme includes recognition that high quality care should be rewarded and that consumers need information to identify the individual providers and hospitals which offer this type of care. But does the clearly desirable goal of promoting high quality health care in turn create disparities?
 
The first step in identifying high quality health care providers is measuring performance. Generally, quality reports capture outcome measures¹ or process measures.²  In 2001, the Centers for Medicare & Medicaid Services (CMS) began a series of quality initiatives designed to improve care to Medicare beneficiaries. Using a standardized measurement system, data are collected and reported on nursing homes, home health agencies, hospitals, and kidney dialysis facilities.³ More recently, CMS established the Physician Quality Reporting Initiative (PQRI), a voluntary program which provides financial incentives to providers who report quality measure data on submitted claims.⁴

Private health plans are also developing quality reporting systems, with mixed reviews. Some physicians question whether a scorecard which tracks specified measures is a true representation of the quality of care being provided to patients.⁵ On the other hand, insurers suggest that providing incentives based on quality scores, as through pay-for-performance programs, will change provider behavior. Moreover, while paying for performance can lead providers to work toward top quality scores, the public disclosure itself is a strong inducement to improve quality and score well. Nevertheless, gathering and reporting quality information on health care providers can generate both intended results and unintended results.

Intended Results

In medicine, as in any profession, there is a natural tendency to work to a higher potential when faced with information about peers who produce a superior result. Thus, the collection and disclosure of quality of care is likely to have a positive effect on providers, including motivating them to use clinical practice guidelines. The information could also serve as the basis for developing quality improvement programs within provider organizations.

As patients have increasingly greater access to information on high performing providers, they will be more likely to seek out those providers.⁶ Over time, variation in the quality of care could be reduced and patients are likely to receive increasingly better care, thereby improving the overall health status of this country. Moreover, referring physicians and health plans may benefit from information on high quality providers.

Unless consumers use comparative quality reports, their value is lost. Thus, educating the consumer on the availability of these reports is paramount. To improve uptake, information must also be easily accessible, understandable, and viewed as trustworthy. While evidence to date does not demonstrate a wholesale realization of the intended consequences of quality reporting, transparency and the fact that providers are more accountable by such disclosures is, in and of itself, a positive step.⁷

Unintended Results

At the opposite end of the spectrum are the potential unintended results which may emerge as quality information is publicized and becomes the basis for rewarding providers. For example, physicians may "practice to the test," meaning they are more likely to focus on measured areas of quality at the expense of important, yet unmeasured areas.⁸ Moreover, physicians, hospitals and other providers may "cherry pick" healthy patients and avoid the sickest and most complex patients in order to improve quality scores. These actions disproportionately affect underserved/minority populations, resulting in an increase in health care disparities. Providers may also avoid certain populations-such as minorities and those with low socioeconomic status-if they believe their performance measures and target goals would be negatively impacted. Ultimately, access to high quality care for these populations could be curtailed.⁹ 
 
The quandary of whether public quality reporting improves quality for all was raised in a debate on public reporting of hospital acquired infection rates. Participants at a symposium of the Infectious Diseases Society of America noted that such reporting "could encourage hospitals and physicians to avoid sicker patients, use intervention targets that may not be appropriate for all patients, and diminish the role of patient preferences and clinical judgment."¹⁰

In a study examining the impact of New York's coronary artery bypass graft (CABG) report cards, researchers noted that while report cards had the potential to improve the quality of health care, "the release of New York's CABG report card was associated with a significant increase in racial and ethnic disparities in CABG use in New York compared with other states in the years immediately after the report card's release."¹¹  After nine years, the disparities returned to pre-release levels, suggesting that over time, physicians may have determined that race was not a good risk indicator or realized that report cards had little relevance to decision-making by patients or referring physicians.¹²

Conclusion

Quality reporting is likely to continue and expand. The challenge for those structuring the systems is to ensure that the intended results of quality reporting are reached, while the unintended results are eliminated. The key is to maintain a rigorous and useful reporting system which does not disadvantage certain populations.

Moreover, quality reporting systems must be designed in such a way as to prevent disparate care. For example, reports should be adequately risk-adjusted to eliminate the likelihood that providers will avoid treating certain groups of patients, thereby exacerbating health care disparities. In addition, by emphasizing measures of appropriate care, physicians are more likely to embrace all patients who would benefit from certain treatments, regardless of their socioeconomic status or race. A focus on process measures, rather than outcome measures may also help reduce the potential for increased disparities insofar as process data is not as closely tied to patient characteristics. Finally, adjusting measures for race and socioeconomic status would help to alleviate the potential for quality reports to create a bias against treating certain patient groups.¹³

For related HCFO-sponsored research, see the grants listed below and see www.hcfo.net.

Title: The Impact of Pay for Performance on Hospitals that Care for Minorities and the Poor
Institution: Harvard University School of Public Health
Principal Investigator: Ashish Jha, M.D., M.P.H.
Grant Duration: February 2008 - January 2009

The researchers will examine the impact of financial incentives to improve quality on hospitals that care for minority or other underserved populations. The Centers for Medicare and Medicaid Services have implemented pay for performance (P4P) demonstrations, and are considering implementing P4P nationally. However, the impact of P4P has not been widely evaluated. Hospitals that care for underserved populations may have greater potential for quality improvement; conversely these facilities lack the tools and resources to improve quality and compete for the additional resources. The researchers will examine changes in quality for hospitals in the Medicare Premier P4P Demonstration that serve disadvantaged populations (minority and poor); these changes will be compared with changes in hospitals in the demonstration that do not serve disadvantaged populations and with hospitals not in the demonstration (and not subject to P4P) that serve disadvantaged populations. The objective of the project is to provide more information about the impact of P4P on hospitals that serve disadvantaged populations, and help policymakers to design incentive systems that encourage higher quality care without disproportionately harming hospitals that care for these populations.

Title: Paying Physician Group Practices for Quality: A Regional Natural Experiment
Institution: University of Washington School of Public Health and Community Medicine
Principal Investigator: Douglas A. Conrad, Ph.D.
Grant Duration: October 2007 - March 2009

The researchers will evaluate the impact of a quality-based scorecard and financial incentives developed by Premera Blue Cross in Washington State. They will compare clinics exposed to two waves of a progressive "paying for quality" intervention with a control group of clinics not subject to the intervention. Specifically, the researchers will assess the joint effects of quality-based financial incentives and the quality scorecard on physicians' clinical quality, patient satisfaction, and efficiency in caring for patients. They will distinguish the effects on quality, patient satisfaction, and efficiency of providing information to medical groups relative to their performance on an array of clinical quality measures from the incremental effect on quality and efficiency of clinical quality-based financial incentives. The objective of the project is to assist organizational leaders and public policymakers to craft more cost-effective quality incentives.

Title: Examining the Quality of Hospital Care and Simulating the Impact of Several Pay-for-Performance Scoring Methods on Hospital Rankings
Institution: Massachusetts General Hospital Institute for Health Policy
Principal Investigator: Joel S. Weissman, Ph.D./Lisa I. Iezzoni, M.D.
Grant Duration: March 2007 - February 2009

The researchers will examine the quality of hospital care. Using patient-level data from a large sample of hospitals collected by the Hospital Quality Alliance (HQA), they will estimate the proportion of patients receiving recommended care, create new measures of patient care quality, and simulate the impact of several pay-for-performance (P4P) scoring methods on hospital rankings. They will also examine the extent to which care varies by race, ethnicity, or insurance status within and across hospitals. The objective of the study is to assist CMS, other public and private payers, and accrediting organizations in developing strategies to improve hospital performance measurement and payment methods, and, ultimately, the quality of patient care.

Title: Involving Consumers in Physician Choice: Making Data into Useable Information for Chronically Ill Patients in Consumer-Directed Health Plans Performance Measurement and Reporting
Institution: Massachusetts General Hospital
Principal Investigator: David Blumenthal, M.D.
Grant Duration: March 2006 - December 2008

What tools will consumers need to help select high performing physicians, within CDHPs? Physician performance data is one of the tools that can be used to help consumers make these decisions. However, there are important opportunities and challenges facing consumer-directed health plans (CDHPs) trying to engage consumers in using physician performance data (PPD). The specific aims of the project are: 1) to develop methods for informing consumers about physician clinical performance; 2) to test the effectiveness of these methods in helping consumers with chronic conditions in CDHPs to make an informed choice of primary care physician (PCP); 3) to explore how consumer characteristics affect their ability to understand PPD and their response to that data. The objective of this study is to understand how and whether PPD can be appropriately and effectively used in CDHPs.

Title: Strategies to Reduce Health Care Providers' Administrative Burden Related to Quality Performance Measurement and Reporting
Institution: Center for Studying Health Systems Change
Principal Investigator: Paul Ginsburg, Ph.D.
Grant Duration: February 2006 - July 2007

How do quality reporting requirements affect hospitals? What strategies do hospitals and quality reporting organizations use to minimize burden? What forces facilitate or impede these efforts? Using a case study approach and building on the HSC's ongoing tracking of local health care markets across the country, the researchers focused on four communities ( Boston, Indianapolis, Seattle, and Orange County, CA ) with a high level of reporting and performance measurement activity. In these communities, the researchers: (1) confirmed the programs that hospitals reported participating in during the Round 5 site visits; (2) confirmed what the reporting requirements are for each program based on background work for the project; and (3) indicated the ways in which hospitals believe reporting requirements of the programs differ enough to meaningfully increase burden. The objective of this study was to explore the burden on hospitals of quality reporting in four communities, extrapolate lessons learned for other communities with similar attributes, and draw implications for policymakers and private sector decision makers seeking to reduce administrative burdens associated with this type of reporting.

Title: Using Physician Profiling Software to Evaluate the Practice Efficiency of Physician Specialists
Institution: University of Southern Maine
Principal Investigator: J. William Thomas, Ph.D.
Grant Duration: July 2003 - June 2004

How does examining the feasibility of using episode-based physician profiling systems help to evaluate the practice efficiency of physician specialists? The researchers completed a HCFO-funded study in which they evaluated the accuracy of seven primary care provider profiling methodologies and examined the implications of differences in accuracy for assessments of physician performance. In this project, the researchers focused on two of the seven methodologies which were episode-based, Episode Treatment Groups (ETGs) and the MEDecision Practice Review System (PRS), to examine 15 (10 medical and 5 surgical) specialties. The objective of the project was to determine whether the risk-adjustment methodologies used to generate reliable profiles in a primary care setting can be extended to specialists given the unique factors that arise in profiling specialty physicians.

Title: The Impact of Performance Reporting on Consumer and Physician Organization Behavior
Institution: Harvard School of Public Health
Principal Investigator: Meredith B. Rosenthal, Ph.D..
Grant Duration: March 2003 - October 2004

How are public "report cards" on consumer and physician behavior being disseminated? The researchers evaluated PacifiCare's Quality Index report cards which provide a relative performance assessment of provider groups in selected areas of clinical, service and administrative quality. The researchers tested (1) how new and continuing health plan enrollees use comparative quality information to select a physician group; (2) how mobilization of consumer choice based on comparative quality information drives physician group performance improvements; and (3) how physician groups are responding to performance measurement when data are used for confidential benchmarking only while other dimensions of quality are reported to consumers. The objective of the project was to fill an information gap concerning the value of publicly reported quality information and to provide guidance to public and private decision makers on the measurement and dissemination of provider quality information.

Title: Assessing the Impact of a Public Report on Hospital Quality: A Controlled Experiment in the State of Wisconsin
Institution: University of Oregon
Principal Investigator: Judith H. Hibbard, Dr.P.H.
Grant Duration: September 2001 - May 2005

How do hospitals react to public reports of their quality and how do such reports influence consumers' perceptions of hospital quality?  This study, conducted by researchers at the University of Oregon, is assessing whether public reports of quality lead to improvement efforts within hospitals. The researchers also are studying whether the public reports create a general impression among consumers about the quality and safety of hospitals in the community. Hibbard and her colleagues are working with The Alliance, a large purchasing group based in Madison, Wisconsin, that will disseminate the public report. The researchers are conducting a controlled experiment in which hospitals are assigned to one of the following three groups. Hospitals in The Alliance, 25 in the region surrounding Madison, will be included in the public report. The remaining 100 hospitals in Wisconsin will be separated by size (large and small) and randomly assigned to either the other treatment group or the control group. The second treatment group will receive a report of their own performance compared with other hospitals that will not be made public. The control group will not receive any reports. The objective of the study is to assess whether public reporting of hospital quality motivated improved behavior and performance and how public reporting affects consumer perceptions of hospital quality.

Title: A Comparative Evaluation of Risk-Adjustment Methodologies for Profiling Physician Practice Efficiency
Institution: University of Southern Maine
Principal Investigator: J. William Thomas, Ph.D.
Grant Duration: May 1999 - April 2002

How accurate are existing physician profiling products used by health plans at predicting/identifying resources used by physicians and physician groups? Researchers at the University of Michigan evaluated these products to answer the following questions: 1) Do some physician profiling risk-adjustment methodologies produce more accurate profiles of physician practice efficiency than others? If so, how do the methodologies compare? 2) How does the number of patients managed by a physician affect the accuracy of the physician's practice efficiency profile? and 3) Are differences in accuracy among profiling systems' risk-adjustment methodologies large enough to affect rankings of physicians' practice efficiency? How consistent are physician practice efficiency rankings from different profiling systems, and how consistent are the systems in identifying outlier physicians? As the researchers noted, physician-profiling information "can be used to select network providers, channel patients, and identify both exemplary practice styles and those that suggest a need for education. Also, reports indicate that profiles are used by health plans for identifying physicians for de-selection from networks." The objective of this study was to evaluate the accuracy of the profiling methodologies being marketed to health plans and examine the implications of differences in accuracy among the tools.

Title: Methods to Present Quality Information to Assist Consumers to Make Health Plan Decisions
Institution: University of Oregon
Principal Investigator: Judith H. Hibbard, Dr. P.H.
Grant Duration: January 1999 - March 2000

How can information on health plan quality be most effectively presented to consumers?  Researchers at the University of Oregon used laboratory studies to examine how consumers process and integrate information in making choices about health plans. While there are many health plan "report cards" available to consumers, little is known about how consumers actually use the information they are given to make decisions. Consumers in a cognitive laboratory environment were presented with different types of information on health plan quality, in different formats, and asked about how they use the information in their decision-making process. The researchers also assessed how specific pieces of information were used and weighted in decisions. The objective of the study was to help determine how best to present information on health plan quality, so that it is valued and understood by consumers, and so that consumers can make the maximum use of information on quality when choosing a health plan.

1 Examples include reports on hospitals' risk-adjusted mortality rates.
2 Examples include AHRQ's National Healthcare Quality Report and NCQA's HEDIS reports.
3 www.cms.hhs.gov/QualityInitiativesGenInfo
4 www.cms.hhs.gov/PQRI/33_2007_General_Info.asp
5 Terry K., "Physician Report Cards: Help, ho-hum or horror?" Medical Economics, July 21, 2006, http://medicaleconomics.modernmedicine.com/memag/article/articleDetail.jsp?id=359028; see also, "Reliability of Current Physician Report Cards Questioned," Press Release, Agency for Healthcare Research and Quality, June 8, 1999 www.ahrq.gov/news/press/pr1999/reptcard.htm
6 However, in a recent Kaiser Family Foundation survey, 2008 Update on Consumers' Views of Patient Safety and Quality Information, only 14 percent of respondents reported that they "saw" and "used" comparative health quality information for health insurance plans, hospitals, or doctors in the past year. This total was down from 20 percent in 2006 and 19 percent in 2004. www.kff.org/kaiserpolls/posr101508pkg.cfm; see also, Tu, T. and J. Lauer. "Word of Mouth and Physician Referrals Still Drive Health Care Provider Choice," Research Brief, Center for Studying Health System Change, No. 9, December 2008 (noting that "For policymakers seeking to engage consumers in provider shopping and quality improvement efforts, a critical challenge is to educate consumers about the existence and the serious implications of provider quality gaps.") www.hschange.org/CONTENT/1028/#ib5
7 Werner, R.M. and D.A. Asch. "The Unintended Consequences of Publicly Reporting Quality Information," Journal of the American Medical Association (JAMA), Vol. 293, No. 10, March 9, 2005, pp. 1239-44.
8 Casalino, L.P., et al. "General Internists' Views on Pay-For-Performance and Public Reporting of Quality Scores: A National Survey," Health Affairs, Vol. 26, No. 2, March/April 2007.
9 Ibid.
10  "Public Reporting of HAIs May Have Unintended Consequences" Infectious Disease Society of America. 2007, www.idsociety.org/PrintFriendly.aspx?id=8066; see also, Costello, P. "Rating Doctors: Who Benefits? As 'report cards' gain favor, some question how far physicians will go to score high," Los Angeles Times, June 13, 2005 www.myhealthcareadvisor.com/news/0050613-latimes
11 Werner, R.M. et al. "Racial Profiling: The Unintended Consequences of Coronary Artery Bypass Graft Report Cards," Circulation, Vol. 111, 2005; pp. 1257-63.
12 Ibid.
13 Werner, JAMA, 2005; Werner, Circulation, 2005