Educational Approached for the Minimum Data Set Series of Instruments

By Dr. John Hirdes, Bonnie Pearson, Nancy Curtin-Telegdi
Many factors contribute to the successful implementation of interRAI's Minimum Data Set (MDS) series of assessment instruments, including good information technology (IT), timely and useful feedback systems and strong leadership at all organizational levels. However, the foundation upon which any implementation effort is built is education. Without effective education of the staff who will actually do the MDS assessments, efforts to implement the instrument and to use its data for decision-making related to clinical practice, resource allocation or the needs of the persons being served can be at risk of failure.

Phillips and Morris (1997) showed that MDS instruments are robust in their reliability and validity in day-to-day use by front line staff in long term care settings. However, Crooks et al. (1997) also showed that the MDS, or any assessment system for that matter, will yield poor information when the instruments are not used appropriately.

Canada faces both tremendous opportunities and substantial challenges as several provinces move toward the implementation of one or more of the MDS instruments. The Commission on the Future of Health Care in Canada (Romanow, 2002) called for a strengthening of the Canadian health care system by expanding the services covered under medicare, improving health information systems and increasing public accountability in health care.

The MDS instruments clearly support the latter two recommendations by providing standardized, comprehensive, clinically relevant information at the individual level that can be used for care planning, resource allocation, outcome measurement and quality management (Hirdes et al., 1999). For example, the Romanow report called for the inclusion of targeted home care services under the Canada Health Act. Data from studies of the MDS-Home Care (MDS-HC) in two provinces have proved helpful in identifying the populations to be served and estimating the resource needs of those populations based on existing practice patterns (Hirdes et al., 2002).

As various provinces and territories implement the MDS-HC, there is an unprecedented opportunity to have better quality information about home care than has ever been available in our country.

The magnitude of the opportunities that can be realized by widespread adoption of the MDS instruments in Canada is mirrored by the challenges posed by their implementation on that scale. Simultaneous educational efforts are underway in entire provinces/territories covering a massive geographic area. There are a relatively limited number of highly qualified educators in the country, and sectors like home care and long term care have relatively few resources for basic service delivery, let alone those needed to deal with large scale organizational changes like the introduction of new assessment systems.

Education is not only needed at the point of introduction of these instruments, but also on an on-going basis to deal with staff turnover and implementation of new applications for MDS data, for example. The temptation to trade-off costs by reducing the proportion of staff members educated in the MDS instruments can compromise the integrity of the data collected, marginalize the clinicians doing the assessments, and increase vulnerability to poor data quality if the organization depends on the skills of a small number of individuals.

The fewer the staff members that are involved in the assessments, the greater the risk that their findings will not be incorporated into day-to-day clinical decision-making. In short, what is needed is a comprehensive, on-going and cost-effective strategy to provide education that ensures high quality, clinically relevant data and supports the transition to evidence-based decision-making at all levels of the organization.

The Learning Needs of Clinicians

The introduction of any new procedure, standards of practice or clinical protocol requires careful consideration of learners' needs. This is particularly true for the introduction of MDS instruments. The information to be learned is complex, the sheer volume of content is substantial, the risk of conceptual overload is high, the pressures on staff to become proficient in a short period of time are great and the receptivity of at least some individuals to organizational change of this scale may be low.

There are a number of different areas of learning that must be covered in any education program related to the MDS instruments. First, there is procedural knowledge that must be gained so that the learner can understand the interRAI approach to assessment (e.g., definitions, process and coding rules for completing individual items; using time frames; inclusion and exclusion criteria).

Second, there is a new language to be learned with implementation of these instruments, including new jargon (e.g., "triggers"), unfamiliar terms related to the various applications of the data (e.g., Clinical Assessment Protocols (CAPs), Quality Indicators (QIs)), and new scales to measure clinical status (e.g., Cognitive Performance Scale (CPS), Depression Rating Scale (DRS)).

Third, the individuals must learn an analytic, problem-solving approach to completing the assessment and using its findings. Since the MDS instruments do not rely on a fixed narrative, but instead depend on multiple information sources, assessors must be able to reconcile multiple inputs that often provide contradictory information. Also, information is not always available from the best possible source, so judgment must be exercised about the accuracy of alternative information sources. Moreover, the use of MDS data to support care planning requires the team to take an investigative approach to using the assessment findings as clues to uncovering underlying problems hindering the optimal functioning of the person being assessed.

Fourth, education programs related to these instruments must also address a host of attitudinal issues that could pose barriers to successful implementation. Without question, the introduction of any new assessment instrument places a substantial burden on the staff that must incorporate it into their daily lives on top of the many time demands they already face. More work is hardly welcome news. In addition, there are numerous misconceptions about standardized assessments (e.g., they reduce "persons" to "tick boxes and numbers") that must be overcome. The effort that will be exerted to complete these assessments must be counterbalanced by a recognition that clients will receive better care because of the insights gained through comprehensive assessment.

Organizational Considerations

One of the primary considerations related to education on any issue in an organization is cost. Some of the obvious costs incurred for both baseline and on-going education can include: replacement time for workers, educational materials and facilities, salary for staff educators and/or consulting fees for external educators, travel costs and registration fees for off-site education.

On the other hand, the less tangible costs of poor, biased or inadequate education can include: compromised data quality; inefficient completion of assessments; reduced impact on the quality of care; inaccurate information resulting in incorrect estimates of case mix and/or quality performance; low staff acceptance; failure to optimize clinical information systems; and treatment of the MDS data as "administrative add-ons", rather than a source of insight into the client's needs and outcomes of care. It is not difficult to imagine that the costs of an effective strategy can be dwarfed by the costs of a poor educational approach.

Educating all clinical staff on the use of the MDS instruments can be difficult in terms of scheduling and overall disruption to the organization. The previously mentioned complexity of these assessment instruments means that education programs must address at least two distinct parts of the learning curve. First, in order to allow the organization to launch routine MDS data collection in a timely manner, there is a need for an intensive introduction to "how to do the assessment". Second, there is a need for continuous learning related to "how to use the assessment".

Staff will typically be best able to engage in more in-depth learning on the applications of MDS data after they have become proficient in completing the assessment. Therefore, organizations must plan for both phases of learning in order to benefit fully from the potential benefits of implementing the instrument. This probably means that a model combining educator-led sessions with self-paced learning that individuals can shape to the demands of their work schedule may be most useful.

Finding the right educator to introduce staff to the MDS instruments can be problematic. There is no certification process available from organizations like interRAI to confirm that a educator has the requisite skills or knowledge or that the content of the education program is appropriate. The training that software vendors provide on how to use MDS software is not the same as the education needed to learn how to do an MDS assessment. To date, health sciences programs in Canada have not incorporated MDS based assessment into the standard curriculum, so one cannot assume that new graduates will be proficient in these instruments.

It is also difficult to find existing educational material that is relevant to the Canadian context and does not place inordinate attention to the case mix or funding applications of the MDS. Conferences, workshops and seminars related to the MDS are offered with increasing frequency, but the costs of sending more than a limited number of staff to these events can be prohibitive.

The international experience in educational approaches for the MDS instruments varies widely (Bernabei et al., 1997; Brunton and Rook, 1999; Landi et al., 1996). Whereas the US has often relied on short, intensive programs that allow very large numbers of persons to be trained very quickly, other jurisdictions (e.g., Italy) have emphasized a longer term educational approach. Perhaps the most widespread trend emerging in jurisdictions facing large scale implementations has been the adoption of the "train-thetrainer" approach. In this model, individuals with relatively advanced levels of expertise in MDS assessment educate others who will in turn become educators within their own organization. This can be helpful because the task of becoming an effective educator can be daunting particularly for someone who is relatively inexperienced.

Educators must not only be proficient in use of the MDS instruments and their applications, but they must also: prepare or acquire appropriate educational materials; anticipate questions from new MDS users and be able to respond to those questions accurately and effectively; identify common trouble spots for new users; manage complex interactions in groups of learners with varying levels of enthusiasm; and problem-solve for unanticipated complications arising with a given site or population. Clearly, new educators will need as much, if not more, support as new MDS assessors.

Outcomes of Successful Education

The design of an MDS education strategy should keep in mind a series of desired outcomes at the individual and organizational levels. Some examples of the desired outcomes for learners include:

From the organizational perspective, many of the education goals would relate to the proportion of learners achieving the abovementioned outcomes. However, one would also hope to see achievement of the following organizational level outcomes:

Learning Preferences for MDS Education

The RAI-Health Informatics Project (RAI-HIP) was a large scale study of the MDS series of assessment instruments that was funded by Health Canada and led by the University of Waterloo, Providence Centre and Lakehead University. Among its substudies was a pilot implementation of the MDS-HC in 14 Community Care Access Centres (CCACs) in Ontario. After the pilot study, a brief survey was sent to participating agencies to obtain case manager feedback on the instrument and on their education preferences. CCACs were invited to seek input from all individuals completing 10 or more MDS-HC assessments in their agency, which resulted in 87 completed surveys. It is important to emphasize that these results cannot be used to generalize to all MDS-HC users, since there were no restrictions on who could participate other than the presence of at least basic experience with the MDS-HC. Therefore, these results should be treated as the results of an open-ended ballot, rather than as an estimate of the prevalence of characteristics in the population of Ontario case managers.

The participants were already at least somewhat experienced users of the MDS-HC, so the relatively low proportion requesting more introductory education (14%) was not surprising (see Figure 1). Among the various applications of the MDS-HC, the greatest level of interest was in more education for care planning (56%), outcome measurement (64%) and quality indicators (62%). Only about 30% of respondents were interested in case mix applications.

The basic options for delivering MDS education include: face to face instruction, conferences, video presentations, print material, and computer based education (CBE). When respondents where asked how they would like to be educated on the MDS, some clear preferences emerged (see Figure 2). Over 80 % of respondents preferred inperson programs, whereas less than 20% preferred CBE methods. The latter finding is not entirely unexpected in the light of age differences in computer use.

Figure 3 shows the proportion of individuals using computers in 2000, based on the Canadian General Social Survey. Computer use declines substantially with age, reaching a low of about 15% in women aged 55 and older. Given that the average age of health care workers continues to rise into the late 40's (CIHI, 2001), it is not surprising that there may be some initial reluctance to embrace CBE methods for this group. In addition, access to computers in health care settings is often limited to administrative staff. Clinicians would tend to have had less experience with computers in the workplace, particularly for educational purposes. This is not to say that the option of using CBE should be ruled out. Rather, it means that CBE programs must be designed carefully to overcome initial hesitance in at least some learners.

An important dimension of effective education is interactivity (Lookatch, 1990). When learners are able to participate through interactive material, there is a greater probability of success. The degree to which the learner can interact with and manipulate the learning material is dependent upon the extent to which user control is embedded in the approach.

For example, unlike CBE methods, video presentations allow virtually no user control beyond stopping, starting, rewinding and scrolling forward through the presentation. The user cannot select the depth of information transmitted, nor can s/he easily navigate the information in a non-linear sequence.

While face-to-face education and conference presentations allow audience members to ask questions and participate in discussions, the learner does not necessarily control: when learning takes place; what is covered or excluded; and the order and depth of discussion. CBE methods allow greater user control over these issues, but they do not typically allow for interaction of the learner with the educator or with other learners. This is sometimes done through Internet chat rooms and listserv discussion groups, but neither of those options truly replicates the collaborative learning that is possible through face-to-face interaction.

Print materials allow the user to control the depth and order in which topics are reviewed, but they lack the interactivity and richness of presentation that can be achieved through in-person, video and CBE approaches. Ultimately, combining these various education methods to take advantage of each of their strengths is probably the most effective strategy.

Instructional Design

Instructional design involves the analysis, design, production and evaluation of educational materials based on principles of learning, in combination with the appropriate use of media (Romiszowski, 1981). A major consideration within the design process is to motivate the learner to: a) engage in the learning experience, and b) persist with the activity in order for learning and retention to occur.

According to the principles of motivational design (Keller, 1979; Dick, Carey and Carey, 2001), instructional materials should be relevant and interesting and they should provide clear expectations for learning outcomes. In other words, the material should be useful for the target audience and it should relate to their work experience. Interest is important because it allows the learner to engage in the education process more effectively, thereby increasing retention and transfer to on the job applications.

Information should be provided in a variety of formats using a combination of text, video, graphics, photographs and animation to accommodate diversity in learning styles (Dunn, 1988). Hidi (1990) showed that interest plays a fundamental role in enhancing learning and attention. By explicitly including learning expectations in educational materials, users can gain confidence and a sense of control of the situation giving the assurance that a certain amount of effort will lead to success in acquiring needed information.

Special Considerations for Adult Learners

Special Considerations for Adult Learners Aside from the previously mentioned considerations related to use of CBE with adult learners, it is important to recognize that education of health care staff entails different considerations than class room based instruction in college or university settings. Although the audience for MDS education will typically be comprised of professionals with higher levels of education than the general population, there is a chance that a portion of the audience will not be recent learners. Therefore, it should not be assumed that everyone will be comfortable or motivated by a standard lecture format as an educational approach.

It is also important to seek out and incorporate learners' experiences into the education process, in part, because it allows them to make the link between what they already know as seasoned clinicians and what they are learning about the MDS. For educational approaches that do not involve in-person interaction, it is particularly important to design the materials to include examples of situations that will be familiar to the target audience.

There is also an important distinction to be made between "learner oriented" and "learning oriented" education (Merril, 1997). A learning oriented approach will include interactivity and responsiveness to learner preferences and interests as design principles, but it will also add structure to allow introduction of new knowledge that even an experienced learner may not be familiar with.

The complexity of learning all facets of the MDS series of instruments clearly demands that learners have the ability to be self-directed and self-paced; however, the educational strategy must also provide sufficient guidance to introduce them to new knowledge that they may not have uncovered without assistance.

Situated Learning

Situated learning allows participants to experience real life circumstances in a way that increases the likelihood of attitudinal and behavioral change (Romiszowski, 1981). Therefore, it is important to allow the learner to experience what it is like to do an MDS assessment as part of the basic educational approach. Simply reading or being told about the assessment is an insufficient method of reaching the desired educational outcomes. There are at least three methods through which situated learning can be incorporated into education sessions: a) learner generated case studies; b) direct assessment of "practice" cases; and c) simulated case studies.

Learner Generated Case Studies

One approach to situating the MDS into the learners' work experiences is to ask audience members to describe one or more case studies of clients they have worked with. This information can then be used to illustrate how the MDS would have been completed for these individuals. This approach can be highly relevant, because it draws directly on the learner's experience. It can also be relatively cost effective, because the exercise can be done without leaving the instructional setting and the time taken for the activity can be controlled by the educator.

The primary drawback of learner generated case studies is that they are unpredictable and provide the educator with less control over the content of that part of the program. For example, not everyone will know the same clients, so the discussion may not include a shared experience and its quality will be heavily dependent on the learner's depth of recall and ability to bring the case to life. The effectiveness of the learning exercise can be severely compromised if the audience is unable or unwilling to provide illustrative cases in a timely manner.

Direct Assessment of "Practice" Cases

The use of volunteer clients to allow new MDS assessors to practice the assessment on them as part of the education program has been a relatively widely used approach. The main advantage of this method is it allows the learner to gain first hand experience doing a real MDS assessment on a real client. It permits the use of different information sources and allows the assessor to conduct his/her first real assessment in a low risk environment. In addition, when several learners assess several different cases, it inevitably leads to a host of issues that will generate useful discussion and problem solving.

Despite the obvious use of situated learning in this approach, there are some serious constraints that limit its usefulness in large scale education efforts. In particular, the organizational challenges of matching learners with persons willing to serve as practice cases, the need to obtained informed consents and the lack of familiarity of external assessors with the care provider are difficult to surmount.

With large group sizes, there may not be enough clients to serve as practice cases, forcing learners to work in groups rather than completing assessments individually. Obviously, this also greatly increases the intrusiveness of the exercise for the client to the point where important ethical concerns arise. In addition, the various organizational demands of using this approach will probably mean that there will be a bias toward less impaired, more compliant and less clinically complex practice cases. This will in turn reduce the ability of the exercise to illustrate some of the more complicated problems one faces with doing the assessment in the field.

Simulated Case Studies

A third option to permit situated learning is the use of simulated case studies developed by the educator. These can be presented in video and/or text form in the instructional setting, and they can be designed as more complex cases to bring out specific points that the educator wishes to discuss in detail. This approach permits the educator to have a more predictable, controlled discussion of the case, and it reduces the time demands for preparing the session. Standardization of case studies also provides a quality control mechanism assuring comparable education experiences for different audiences. This approach is also substantially less disruptive for the organization and it does not involve intrusion on client privacy.

The use of simulated cases comes at the expense of realism for the learner. Cases that are designed to illustrate variations in as many MDS items as possible may appear too artificial to resonate with the learners' experiences. This problem can be exacerbated by the need to fill in gaps in the assessment that are not already discussed in the case study. Moreover, if only a single case study is used, it may leave learners' with the impression that the MDS instrument applies only to a narrow range of clients.

Concluding Comments: Choosing the Right Education Strategy

There is no singular, gold standard model for how MDS education should be done. One might expect a broad range of educational resources to emerge to fill this need in the Canadian market place over the next few years. The choice of what strategies to use must consider the instructor's skills, experience of the learners, time demands, and financial resources available. A flexible approach that uses multiple educational methods to address different needs at different points in the learning curve will probably be most successful. What is certain is that effective education requires an organization to commit appropriate resources to meet the learning needs of its staff. The introduction of MDS instruments can have profound positive effects on an organization's capacity to make evidence based decisions. The quality of education will have a direct impact on the quality of the data upon which those decisions will be based.

References

Bernabei, R., Murphy, K., Frijters, D., DuPaquier, J.N., Gardent, H. (1997). Variation in Training Programmes for Resident Assessment Instrument implementation. Age and Ageing 26[suppl. 2]: 31-35.

Brunton, B., Rook, M. (1999). Implementation of the Resident Assessment Instrument: A Canadian Experience. Healthcare Management Forum 12: 49-53.

Canadian Institute for Health Information. (2001). Canada's Health Care Providers. Ottawa: CIHI.

Crooks, V.C., Schnelle, J.F., Ouslander, J.P., McNees, M.P. (1995) Use of the Minimum Data Set to Rate Incontinence Severity. Journal of the American Geriatrics Society, 43: 1363-1369

Dick, W., Carey, L., & Carey, J. (2001). The Systematic Design of Instruction. 5th Edition. New York, Longman.

Dunn, R. (1988). Teaching students through their perceptual strength or preferences. Journal of Reading, January.

Hidi, S. (1990). Interest and its contribution as a mental resource for learning. Review of Educational Research, 60: 549-571.

Hirdes, J.P., Fries, B.E., Morris, J., Steel, K., Mor, V., Frijters, D.H. et al. (1999) Integrated health information systems based on the RAI/MDS series of instruments. Healthcare Management Forum 12: 30-40.

Hirdes, J.P., Poss, J., Fries, B.E., Morris, J.N. (2002) Consultants' Estimates on Costs of Targeted Home Care. In Romanow, R.J. Building on Values: The Future of Health Care in Canada, Ottawa: Commission on the Future of Health Care in Canada, 325-330.

Keller, J. (1979). Motivation and Instructional Design: A Theoretical Perspective. Journal of Instructional Development, 2:26-34.

Landi, F., Sgadari, A., Cipriani, L., Castellani, M., Pahor, M., Carbonin, P.U., et al. (1996). A Simple Program to Train Case Managers in Community Elderly Care. Aging (Milano) 8: 211-218.

Lookatch, R. (1990). How to Talk to a Talking Head. Training and Development Journal, September, 63- 65.

Merrill, M.D. (1997). Learning-Oriented Instructional Development Tools. Performance Improvement, 36: 51-55.

Phillips, C.D., Morris, J. (1997). The potential for using administrative and clinical data to analyze outcomes for the cognitively impaired: an assessment of the Minimum Data Set for nursing homes. Alzheimer Disease and Associated Disorders 11: 162-167.

Romanow, R.J. Building on Values: The Future of Health Care in Canada, Ottawa: Commission on the Future of Health Care in Canada, 325-330, 2002.

Romiszowski, A. (1981). Designing Instructional Systems: Decision Making in Course Planning and Curriculum Design. New York: Kogan Page, London/Nichols Publishing.

Wilson, A. (1993). The Promise of Situated Cognition. New Directions for Adult and Continuing Education, 57: 71-79.