PACE
Developing an Outcome-Based Continuous Quality Improvement
System for the Program of All-Inclusive Care for the Elderly (PACE)
(Outcome-Based CQI System for PACE)
Funded by: Health Care Financing Administration
Project Period: 9/96 - 9/04
Abstract
Objectives: To develop and empirically test the effectiveness
of a set of outcome measures and corresponding data items (Core Outcome and
Comprehensive Assessment [COCOA] data set) designed uniquely for monitoring and
improving (where appropriate) outcomes of care provided to Program of All-Inclusive
Care for the Elderly (PACE) participants across the United States.
Methods: Developmental activities included the specification
of outcome indicators, outcome measures, and data items based on extensive review of
the geriatric and gerontologic literature; review of PACE programmatic information;
focus groups; and multiple clinical and research panel reviews involving over 100 PACE
care providers. To ensure the validity of the outcome measures and the feasibility,
reliability, and validity of the COCOA data items, a three-phase empirical field test
was conducted. An initial feasibility test was completed at two PACE sites with a
sample of 30 participants. Interrater reliability for COCOA data items was estimated
using repeat assessments of two different PACE staff members within a 24-hour period
for a sample of 80 PACE participants at three sites. Validity of the outcome measures
(e.g., sensitivity to site-level differences) was assessed based on data collected
during a six-month systematic field test (SYFT) for a sample of 1799 PACE participants
at 13 PACE sites. During each month of the SYFT, participants due for (re)assessment
were assessed using the COCOA-B data items. Participants were assessed every three to
four months during the SYFT.
Results: The COCOA data set resulting from the developmental
activities of the project included 212 data items. The number of data items was
reduced substantially based on findings from the first two phases of the field test,
resulting in a set of 133 items (necessary to compute the 44 outcome measures most
relevant to the PACE community) implemented during the SYFT. The majority of the COCOA
data items yielded acceptable interrater reliability (kappa >0.60). COCOA items with a
reliability coefficient of less than .60 were revised or eliminated from the data set
prior to the SYFT. SYFT findings showed that outcome measures related to physiologic
functioning and participant and informal caregiver quality of life demonstrated greater
between-site variability than those measures related to functional status and
participant and informal caregiver satisfaction.
Discussion: The multiphase field test demonstrated that it is
feasible to integrate the COCOA data set and data collection protocols into PACE site
assessment routines. One limitation of the project was the brief time frame allocated
for the SYFT. Further work needs to be done to refine the outcome measures
(particularly those related to functional status) to capture change over an extended
period of time (e.g., 12 months). The set of proposed outcome measures and the
associated COCOA data set produced by this project can be used to provide a data
system for quality monitoring by the Centers for Medicare & Medicaid Services and
outcome-based continuous quality improvement efforts by PACE sites.
Key Staff: Kaehny, Donelan-McCall, Bontrager, Braudrick, Crisler, Engle, Floersch,
Haarhues, Kramer, McCullen, Schlenker, Shaughnessy
Research Monographs:
Kowalsky JC, Kutner J, Kramer AM, Kaehny MM, Hatfield J, Fernández-Fletcher M. "Deliverable 6: Summary of Findings from Focus Groups". Center for Health Services and Policy Research, Denver, CO. November 1997.
Kaehny MM, Belansky ES, Kramer AM, Schlenker RE, Crisler KS, Fernández-Fletcher M, Eby CA. "Outcome Measure Development, Specification of Data Items, and Design of Feasibility and Pilot Tests". Center for Health Services and Policy Research, Denver, CO. March 1999.
Mottram K, Kaehny MM, Crisler KS, Talkington S, Schlenker RE, Kutner J, Bush A. "Product 2: Summary of PACE Assessment Approaches". Center for Health Services and Policy Research, Denver, CO. December 1999.
Jordan AK, Mottram K, Wolgemuth J, Talkington S, Kaehny MM, Crisler KS, Kramer AM, Kutner J, Schlenker RE. "Product 3: Assessment Literature and Related Programs". Center for Health Services and Policy Research, Denver, CO. December 1999.
Brega AG, Kaehny MM, Talkington S, Schlenker RE, Crisler KS. "Product 4: State and Federal Data Requirements for PACE". Center for Health Services and Policy Research, Denver, CO. December 1999.
Kaehny MM, Donelan-McCall N, Mottram K, Crisler KS, Schlenker RE, Shaughnessy PW. "Product 6: Initial Preliminary Draft of the Integrated COCOA/OBCQI Data Set, Related Developmental Activities, and Plans for Field Testing". Center for Health Services and Policy Research, Denver, CO. July 2000.
v