What is Computer Adaptive Testing (CAT)? Why is CAT a “smart” and efficient method for electronic outcomes collection?
All electronic outcomes measures are not created equal! Valid, reliable, responsive and risk-adjusted outcomes data is important for accurately establishing your value as an efficient and effective clinician. The ability to show objective evidence of the quality of your care to patients, physicians and payers depends on the outcomes you use and the strength of the system measuring your outcomes.
To maximize your ability to show your service was effective and efficient, you need tools that are responsive to patient change. Although many legacy outcomes can be taken in electronic form, the tool is the same as the pen and paper version. The psychometric properties of these assessments do not change just by using technology as the medium for administration. For example, the Oswestry Disability Index, the DASH, the Neck Disability Index, and the Lower Extremity Functional Scale (LEFS) can be completed on a computer or a tablet. All technology has done is saved a tree and saved your time scoring the responses.
What sets FOTO’s electronic outcomes measures apart from other electronically collected measures?
FOTO’s “Smart” assessments use Computer Adaptive Testing (CAT). FOTO’s “smart” assessments are body part and impairment specific and are customized to each patient. Each assessment begins by asking the patient about a functional activity that is of moderate difficulty and immediately adjusts what next question will be asked based on the patient’s previous question response. This means the patient only answers 4-6 relevant questions and it takes less than five minutes on average to complete and for the CAT to determine the patient’s functional level. This makes the assessment much more efficient and meaningful for the patient.
What is the science behind the “smart” assessments? How exactly do the CATs work?
CAT uses a calibrated item bank of questions that measures patient function. The item bank includes activities ranging from higher to lower function.
Item Response Theory
CAT uses the Item Response Theory (IRT) to estimate function and select the most appropriate items. IRT math is what is used to calibrate the questions within an item bank. This results in each question having its own assigned place on the metric from low to high functioning.
Entry Level Question
At the beginning, the CAT does not know anything about the patient, so it starts by assuming that the patient is of average ability. Each item bank has
a specific average ability question that has been pre-assigned as the entry level question. For example, in the Shoulder CAT, the entry level question asks about the ability to reach a shelf at shoulder height. Once the patient has answered the entry level question, the CAT starts to get information about the patient’s ability and calculates an initial estimate of the patient’s functional status (FS).
Selection of Subsequent Questions in the CAT
The CAT uses the information gained from the patient’s response to the entry level question to select the next question. After each question, the CAT adjusts what next question will be asked based on the patient’s previous response. The CAT process tries to find the precise level of function by asking questions about activities of slightly higher or lower function.
The score the patient receives at the end of the questionnaire immediately translates to how the patient is functioning. Through this process, CAT individualizes patient scores.
When has the CAT gathered all the information it needs?
Stopping Rules: After each question is answered, CAT updates its estimate of the patient’s functional level. The CAT continues to ask questions until 1 or more of the stopping rules are satisfied. The rules are:
- No change in FS estimate over a certain number of consecutive question responses (usually 3).
- Pre-determined level of low standard error has been calculated.
Once one of these rules is met, the CAT produces a final functional status score. In this way, the stopping rules insure that the functional status score is precise. Scoring
FOTO CAT measures are typically reported on a scale of 0-100 where a higher score indicates higher function. The score of 0 and 100 is not a percentage of function.
Excerpts from this post originally appeared in FOTO’s Rehab Outcomes Blog, written by Selena Horner, PT.