Item/test information estimation fails

Hello, I accidentally discovered a failure in item and test information estimation in constrained GPCM model. I estimated a Rating Scale Model, but with a slope freely estimated (while thresholds constrained across items; I call it here simply RSM). I am aware of this type of model is problematic, but this does not change the fact that information is estimated wrongly.

Here is the code. I estimated both the GPCM and the RSM models. Some items performed well (e.g., item 1), but some others not (e.g., item 3). The estimated item info (and thus the total test information and SEs returned in plot and itemplot functions) are implausible.

dat <- ShinyItemAnalysis::HeightInventory

## Generalized partial credit model
gpcm1 <- mirt(dat[dat$gender == "F", 1:26], 1, itemtype = "gpcmIRT", SE = T) ## females only

## Generalized rating scale model
mod1 <- "
F1 = 1-26
CONSTRAIN = (1-26, b1), (1-26, b2), (1-26, b3)
FREE = (2-26, c)
"
grsm1 <- mirt(dat[dat$gender == "F", 1:26], mod1, itemtype = "gpcmIRT", SE = T)

## Item trace lines and information functions
itemplot(gpcm1, 1, type = "infotrace", main="GPCM, item 1, works well")
itemplot(grsm1, 1, type = "infotrace", main="GRSM, item 1, works well")

itemplot(gpcm1, 3, type = "infotrace", main="GPCM, item 3, works well")
itemplot(grsm1, 3, type = "infotrace", main="GRSM, item 3, item info fails")

itemplot(grsm1, 3, type="infoSE", main = "Note that errors are biased too as they cannot be zero.")

Well, I can verify the results using fscores procedure. This is unbiased and results are plausible (I suppose it uses Hessian matrix to estimate SE, while itemplot uses item information).

## ex-post item info for item 3, GRSM model
grsm1_sc <- as.data.frame(fscores(grsm1, full.scores.SE = T)) 
grsm1_sc$info <- 1/grsm1_sc$SE_F1**2 ## estimates test information from SEs
plot(grsm1_sc$F1, grsm1_sc$info, xlab = "theta", ylab = "information", 
     main = "Test information estimated from SE")
plot(grsm1_sc$F1, grsm1_sc$SE_F1, xlab = "theta", ylab = "standard errrors", 
     main = "Standard Errors")
plot(grsm1, type="infoSE", main = "Note that errors are biased too as they cannot be zero.")

I haven't checked the issue further, for example, if the same problem occurs with Rasch RSM model too. Unfortunately, the issue might have some consequences in CAT etc.

philchalmers / mirt

Item/test information estimation fails #244