A 70% error rate is unacceptable for assessing quality.
An evaluation process with that level of error is not only unacceptable, but also reflective of irresponsibility of negotiators to ensure some level of accuracy in the process. This level of error represents far more than just an “oops.”
When the distribution of outcomes is divided into 10 equal groups, unadjusted versus risk adjusted outcomes matters. Without risk adjustment, 70% of the time, the outcome was categorized into a different group than a risk adjusted outcome.
In order to be more accurately evaluated on outcomes and quality, patient characteristics need to be considered and factored into the evaluation. Without risk adjustment that takes into consideration patient characteristics that affect outcomes, a high amount of error happens which affects a clinicians ranking of performance.
You’ll find the abstract to the recent study below.
Impact of Risk-Adjustment on Provider Ranking for Patients With Low Back Pain Receiving Physical Therapy.
Study Design: Retrospective cohort.
Background: Impact of risk-adjustment on clinic quality ranking for patients treated in physical therapy outpatient clinics is unknown.
Objectives: To compare clinic ranking, based on unadjusted vs. risk-adjusted outcomes, for patients with low back pain (LBP) treated in physical therapy.
Methods: Secondary analysis of data from adult patients with LBP treated in outpatient physical therapy clinics during 2014-2016. Patients with complete outcomes data at admission and discharge were included to develop the risk-adjustment model. Only clinics with complete outcomes data for at least 50% of patients, and 10 or more complete episodes of care per clinician per year, were included for ranking assessment. R-squared shrinkage and predictive ratio were used to assess for overfitting. Percentile ranking by deciles, or three distinct quality ranks based on uncertainty assessment, were used to assess agreement between unadjusted and adjusted rankings.
Results: The primary sample included 414,125 patients (mean age (SD) =57(17); 60% women) treated by 12,569 clinicians from 3,048 clinics from all US states; 82% of patients from 2,107 clinics were included in the ranking assessment. R-squared shrinkage was less than 1% with a predictive ratio of 1. Risk-adjustment impacted ranking for 70% or 31% of clinics based on deciles or three distinct quality levels, respectively.
Conclusion: Important changes in ranking were found after adjusting for basic patient characteristics of those admitted to physical therapy for treatment of LBP. Risk-adjustment profiling is necessary to more accurately reflect quality of care when treating patients with LBP. Level of Evidence Therapy, level 2b.
J Orthop Sports Phys Ther. 2018 May 22:1-35. doi: 10.2519/jospt.2018.7981. [Epub ahead of print]