rbcavanaugh
commented
2 years ago I think the thing to do would be to compare the ability estimates and sem after each response between pnt-cat and catpuccino. If those are also different perhaps something is wrong.
I think the pnt cat estimates are based on a larger sample of pwa than catpuccino though right? Could that potentially change the item difficulty estimates?
If this is the case, then when Will sends me the scores from his revised estimates using the Stan code, I can check out app to make sure the estimates are right.
william-hula
It occurred to me this morning that the pnt-cat30 is delivering items differently than the pnt-cat30 when the participant gets the first couple correct, such that microscope is the third item delivered.
The order when a participant gets all items correct from the anomia R03 for the first 10 items is:
pumpkin, ruler, cheerleaders, volcano, microscope, stethoscope, pyramid, binoculars, dinosaur, and ambulance.
However, in the PNT-CAT they are: pumpkin, ambulance, microscope, stethoscope, pyramid, binoculars, volcano, dinosaur, cheerleaders, thermometer.
Just making a note of this because it could mean a couple things: 1) are item choice algorithm is different than catpuchino 2) our item difficulty parameters have changed 3) An error occurred somewhere in the t-score transformation 4) something else.
@william-hula @rbcavanaugh