Improving Nursing Home Quality of Care Through Outcomes Data The MDS Quality Indicators

By Dr. David Zimmerman

The quality of care provided to nursing home residents is a continuing source of concern on the North American continent, and throughout the world. As Dr. John Hirdes noted in a previous article in STRIDE (February-May, 2000), Canadians “have become much less tolerant of poor quality of care”.

Nursing home quality also has been a long-standing issue in the United States, where a study commissioned by the Institute of Medicine more than 15 years ago found that there were serious deficiencies in nursing home care across the country (IOM, 1986).

A subsequent IOM-commissioned study last year found that serious problems still existed (IOM, 2001). One key result of the earlier study was the enactment and implementation of the Nursing Home Reform Act of 1987, a provision of which was that the federal government develop and mandate the use of a standardised resident assessment instrument, called the Minimum Data Set (MDS). The MDS was developed and implemented several years later, and all nursing homes in the United States are required to use the MDS in the resident assessment process. Through the efforts of Dr. Hirdes and the InterRAI group in Canada, there is extensive use of the MDS here as well.

Development of the MDS Quality Indicators

The MDS Quality Indicators were developed by a team of researchers at the Center for Health Systems Research and Analysis (CHSRA) at the University of Wisconsin-Madison (Zimmerman, et.al. 1995). They were developed as part of a national demonstration to develop and test both a payment system and quality monitoring system based on the resident level data in the MDS. This involved extensive interdisciplinary clinical input, empirical analyses, and field-testing. Clinical and research staff at the University of Wisconsin-Madison developed an initial draft of a set of indicators and potential associated risk factors, which were based on an extensive review of relevant clinical research literature and the care-planning guidelines.

Out of this review came a proposed set of indicators covering 12 domains, or areas of care. This initial set of indicators was reviewed by national clinical panels representing the major disciplines involved in the provision of nursing home care, including nursing, medicine, pharmacy, medical records, social work, dietetics, physical therapy, occupational therapy, and speech and language therapy, as well as resident advocates and administrators. The panels provided a rigorous critique, assisted in refining or deleting proposed quality indicators, and suggested new quality indicators. From an initial set of 175 quality indicators representing the 12 domains of care, the QI’s underwent considerable further study to determine their clinical validity, feasibility or usefulness of the information, and statistical robustness.

As a result of these analyses, the number of quality indicators was reduced to approximately 100, and then further to 31 indicators. These were then subjected to series of pilot tests to determine their utility and feasibility in guiding the nursing home quality assurance process (called the “survey process” in the U.S.) and additional revisions were made (Karon and Zimmerman, 1996).

The revised set of indicators was then the subject of validation testing, described below.

The final revision of the quality indicators was necessitated by the need to make the indicators compatible with the version of the MDS instrument that was mandated for use throughout the country. In the United States, a complete MDS assessment is required of residents upon admission, significant change in status, and annually. Assessments are completed every three months, but the facilities need only collect a subset of data. As a result, it was necessary to modify the quality indicators to accommodate the more limited scope of quarterly data, which resulted in a further reduction to 24 quality indicators.

Quality Indicator Characteristics

Quality indicators provide information on the presence (or absence) of selected care processes and outcomes. Some indicators show change over time (called “incidence measures”), while others represent status at a point in time (called “prevalence measures”). Information is calculated at the resident level and aggregated to represent care within a given facility. At the resident level, quality indicators are defined as the presence or absence of the condition, based on data from the most recent MDS assessment. For a single facility, the quality indicator is defined as the proportion (percent) of residents with the quality indicator condition. These facility level quality indicators can be used to compare any given facility with others or with nursing home population norms.

Quality indicators capture both processes and outcomes of care. Process indicators represent the content, actions and procedures invoked by the provider in response to the assessed condition of the resident. Process quality includes those activities that go on within and between health professionals and residents. Outcome indicators represent the results of the applied processes, as inferred from resident status.

Outcome quality addresses questions of how the resident fared as a consequence of the provision of care, i.e., whether the resident improved, remained the same, or declined. In some cases, the quality indicator is a combination of an outcome and a process, in that it reflects both of them. An example is the quality indicator for the presence of symptoms of depression (outcome) with no treatment (process) indicated.

Adjusting the Indicators for Resident Risk

Some quality indicators have associated “risk adjustment factors” that improve the ability to make fair comparisons across residents or facilities. An important issue in developing measures of quality is the need to adjust for variation in the risk of negative outcomes. We conceptualised risk as the probability that, given a particular combination of health or functional conditions, a resident will require certain care processes or will experience specific adverse health outcomes (Arling, et. al, 1997). Ideally, quality assessment systems should distinguish adverse outcomes that are “avoidable” or that result from the quality of resident care, from those that are “unavoidable” or that follow from the natural course of disease or disability.

The purpose of risk-adjustment is to remove effects of resident risk over which the facility has no control from those factors reflecting the quality of care provided by the facility. Thus, facility comparisons based on risk-adjusted QI's should be a more accurate reflection of care quality than unadjusted indicators, which should, in turn, result in more effective targeting of quality of care problems, at both the facility and resident levels (Maxwell, et. al., 1998).

There are two approaches to risk adjustment that have been used most often in the nursing home context. The QIs developed by CHSRA and currently in use in the U.S. use a stratification approach. Under this method, individual residents’ risk factors are identified, and each resident is classified as “high” or “low” risk. The facility’s QI scores are then calculated separately for each of the risk groups. Cross-facility comparisons are made within specific risk categories.

A second method uses a regression-based approach to risk adjustment. Under this approach, a regression model is developed to predict each individual’s likelihood of experiencing the outcome of interest. Facility QI scores are calculated by comparing the percentage of residents who have the QI condition to the percentage of residents that would be predicted by the statistical analysis.

There has been considerable discussion and debate over the preferred risk adjustment approach. The ultimate decision on which method has the most utility must be made on the basis of the statistical and practical superiority of the two methods, which, in turn, must depend on the purposes for which one wishes to risk adjust. With respect to the statistical criteria, the case must rest on the relative sensitivity and specificity of the two methods, which requires appropriate validation of the quality indicators using a gold standard criterion against which to compare their predictive ability.

Short of that, it is instructive to examine the amount of change in the QI distributions (from unadjusted to adjusted status) that results from each method. In side-by-side comparisons, the stratification model has been shown to have produce greater differences and more movement from unadjusted to adjusted status than the regression approach (Karon and Zimmerman, 2001).

The CHSRA MDS Quality Indicators

The CHSA MDS Quality Indicator set is comprised of 24 Quality Indicators, all of which are based on resident level data from the MDS. These indicators represent 11 of the original 12 domains of care. The 24 quality indicators are presented in Table 1, which also classifies each quality indicator as process or outcome, and whether the quality indicator has associated risk factors.

Table 1 - Quality Indicators (Version 6.2) - Defined on Basis of MDS 2.0 with Partial Quarterly Assessments [available in the print version of Stride]

Validation of the Quality Indicators

It is critically important to conduct studies of the accuracy and validity of any quality measures. However, because of funding cutbacks on the aforementioned national demonstration that led to the their development of the indicators, only a limited validation study of the quality indicators could be undertaken prior to their application in the quality assurance process (Karon and Zimmerman, 1996). Findings from the validation study showed, first, that the quality indicators have a high degree of accuracy, or reliability. Average facility accuracy rates for the Quality Indicators ranged from 72 per cent to 95 percent.

The findings also reveal that the Quality Indicators have reasonably high predictive power at higher threshold levels: if a facility flagged at the 90th percentile, the probability that follow up review found a problem with care is almost 70 percent, while the corresponding probability at the 95th percentile rises to 88 percent. In general, the validation study findings showed that the Quality Indicators are useful tools for identifying quality problems related to all aspects of the care process (i.e., assessment, care planning, implementation, and monitoring of care).

Using the Quality Indicators in the United States

Quality indicators have several important applications. They can be used as part of an external quality assessment or review process; as part of a provider’s internal quality improvement activities; as the basis of research into care practices; as a source of consumer information; and to help guide policy makers. In the United States, the CHSRA MDS Quality Indicators are used in the federal and state quality assurance systems in all 50 states, and reports on the indicators are available to all 17,000 nursing homes for use in their internal quality improvement initiatives.

Researchers at CHSRA developed a set of quality monitoring reports that can be used by external and internal quality assurance staff. These reports enable the reviewer to compare the facility with other facilities; and at the resident level, to identify those residents who have specific conditions reflected in the QI’s.

The MDS Quality Indicators have also been as the basis for many internal quality improvement projects. One example of such a program is the CHSRA Provider Initiative Project, which offers nursing home staff assistance in understanding and interpreting the quality indicators, as well as a vehicle for submitting MDS data and receiving reports on the Quality Indicators. In addition, supplementary materials such as specific QI protocols to use in reviewing care areas that flag on the QI reports are provided to participating nursing homes. More generally, the emphasis on quality and outcomes management has led to the increased use of MDS based performance measures by managed care organisations and private consultant firms in the long term care area. Outcome based performance measures are more and more the cornerstone of quality improvement efforts, both in nursing homes and in other long term care settings.

Where Do We Go From Here?

The availability of the MDS has made possible the development of resident-based quality indicators. These indicators provide important information that can be used for many purposes, including internal quality improvement initiatives. Other important extensions of this concept can be identified, and there are other audiences for whom the Quality Indicators, and nursing home performance measures more generally, can be extremely valuable.

Perhaps of greatest potential value is the systematic use of such measures by nursing home residents, their families and informal caregivers, consumers, and advocates for nursing home residents. Use of this information can enhance the ability of these groups to make more informed decisions about the selection of a formal caregiver, and to more effectively monitor - as well as participate in - the care they provide. There are currently several initiatives underway in the United States to utilize MDS-based quality indicators in public reporting of nursing home performance. The Centers for Medicare and Medicaid Services (CMS—formerly known as HCFA) is testing the use of quality indicators on public websites in six states, and CMS has launched a process to get consensus on the use of a set of indicators nationally before the end of 2002. Several states already have initiatives using publicly reported MDS-based quality indicators.

With respect to provider initiatives, quality indicators can help to identify potential care areas to focus on, identify specific residents to key on in the review and investigation process, assess the resulting magnitude and the nature of the care problem, and provide a structure and organising framework for that process. A systematic response to the identified problems is then needed. Other sources of assistance are available and internally, focus groups of staff, as well as residents and family members, can formulate strategies to serve as the basis for quality improvement projects. The availability of universal data from a standardised assessment tool can also facilitate progress in clinical care practices through more robust epidemiological studies.

There is considerable activity at the federal level in the United States to (a) improve the quality indicators and measures for a variety of purposes, (b) expand the concept of quality indicators to other care areas, populations, and settings, and (c) utilize the available data more effectively in quality assurance programs. Through contracts with researchers, CMS is currently:

These initiatives offer testament to the acceptance of quality indicators based on resident assessment data and to the belief that they have enormous potential for improving care by providing better information to a variety of users, including providers of care, consumers, regulatory agencies, and the research community.

References

Arling G, Karon SL, Sainfort F, and Zimmerman DR. “Risk Adjustment of Nursing Home Quality Indicators,” in The Gerontologist, 37(6), 1997.

Hirdes, JP. “Long-Term Care in the Information Age: The Potential of the MDS”, in STRIDE, February- May, pp 14-15. 2000.

Institute of Medicine. Improving Quality of Care in Nursing Homes. Washington, D.C. National Academy Press. 1986.

Institute of Medicine. Improving the Quality of Long Term Care. Washington, D.C. National Academy Press. 2001.

“Risk Adjustment: Why Bother?” S.L. Karon and D. Zimmerman. Presented at the First Joint U.S. and Canadian Case Mix and Quality Assurance Conference, Niagara Falls, Ontario. October 29, 2001.

Karon SL and Zimmerman DR. “Using Indicators to Structure Quality Improvement Initiatives in Long- Term Care,” Quality Management in Health Care, 4(3), 54-66. 1996.

Maxwell C, Zimmerman DR, Karon SL, and Sainfort F. “Estimating Quality Indicators for Chronic Care from the MDS 2.0: Risk Adjustment Issues and Concerns.” Canadian Journal of Quality in Health Care Vol. 14(3): pp.4-13, 1998.

Zimmerman DR, Karon SL, Arling G, Ryther-Clark B, Collins TM, Ross R, and Sainfort F. “The Development and Testing of Nursing Home Quality Indicators”, Health Care Financing Review, Vol. 16, No. 4, Summer 1995.