search

rbcavanaugh / pnt

Beta-version of R Shiny implementation of the computer-adaptive Philadelphia naming test (Roach et al., 1996) following item response theory.
https://william-hula.shinyapps.io/pnt-cat/
GNU General Public License v3.0
0 stars 6 forks source link

Should the app provide an option where the next item is randomly selected from 2-3 best options? #9

Closed rbcavanaugh closed 3 years ago

rbcavanaugh commented 3 years ago

see title.

gfergadiotis commented 3 years ago

Hey Robert, can you clarify what happens to the 2-3 options that are not selected? Are they left in the item bank? Or, are they taken out of the available items for that administration, so that they are available for future administrations? I suspect it is the latter, given that that is a strategy for ensuring that not all items in a given difficulty/ability region are exhausted in the first administration. In principle, I would agree with this strategy. However, our published work has not used that feature in the past.

~ Will, it would be awesome if we tested this in the 2PL sim study. Along with providing informed θ's based on prior administrations.

rbcavanaugh commented 3 years ago

what happens to the 2-3 options that are not selected?

Could be either - I don't have a preference. Both are doable from a coding stand point. I'd leave this up to the experts.

william-hula commented 3 years ago

Rob, what was your original motivation for asking the question? My guess is that implementing this item selection strategy won't make a lot of difference for the basic PNT item bank, and if it does, it'll just decrease the average length of the follow-up CATVL at the expense of lower precision for the initial CAT30. It might have a bigger impact and be more useful with the larger item bank that we'll have in a few years.

rbcavanaugh commented 3 years ago

yes. I believe this came up in the context of repeat administrations. I'm happy to leave it as is right now, and return to it when the item bank is much larger.

rbcavanaugh commented 3 years ago

closed - will revisit later on if needed