The differences in cov matrix result in significant variations in the simulation outcomes of the mirtCAT study.

[Hsin21]

Hi, recently I've been conducting multidimensional (seven dimensions) simulation studies using the mirtCAT package. The purpose is to compare the simulated testing conditions with real-life condition to determine their alignment. Depending on the context, I employed both randomly generated cov matrix and cov matrix obtained from the database to generate response data for the participants using these two distinct approaches. And I utilized these two sets of response data separately to do simulate study .

However, I encountered an issue with the results of the simulation study. The significant discrepancy between the two simulations is concerning. Using data generated from a randomly generated cov matrix, only about 37 items were required for the CAT, while using data from a real cov matrix necessitated 88 items. I believe that such a significant difference shouldn't arise solely from differences in cov matrix. Therefore, I am wondering whether the package might encounter issues with certain types of cov matrix?

It would be really helpful if you could assist in checking if there is any issue. Thank you very much.

The following is my code : ``

randomly condition

N <- 10000

a_params <- item.params$items[, c("a1", "a2", "a3", "a4", "a5", "a6", "a7")] a_params d_params <- item.params$items[, c("d1", "d2", "d3", "d4", "d5")] d_params

mu <- c(0, 0, 0, 0, 0, 0, 0)

set.seed(2023)

matrix_size <- 7

random_cov_matrix <- matrix(runif(matrix_size^2, -1, 1), nrow = matrix_size) random_cov_matrix <- random_cov_matrix %% t(random_cov_matrix) diag(random_cov_matrix) <- 1
random_cov_matrix[random_cov_matrix > 1] <- 1 random_cov_matrix[random_cov_matrix < -1] <- -1 random_cov_matrix <- nearPD(random_cov_matrix, conv.tol = 1e-7, corr = TRUE)$mat random_cov_matrix <- 0.5 (random_cov_matrix + t(random_cov_matrix)) random_cov_matrix <- as.matrix(random_cov_matrix)

print(random_cov_matrix) true.theta <- rmvnorm(N, mean = mu, sigma = random_cov_matrix)

simulated_data <- simdata(a = a_params, d = d_params, N = N, itemtype = 'graded', Theta = true.theta)

design2 = list(min_SEM = c(0.4,0.4,0.4,0.4,0.4,0.4,0.4), max_items = 98)

CAT simulation

trysim <- mirtCAT(mo = sim, local_pattern = simulated_data , method = 'MAP', criteria = "Trule", design = design2)

real condition

mu <- c(0, 0, 0, 0, 0, 0, 0)

true_cov_matrix <- matrix(c( 1.000, 0.126, 0.131, 0.382, 0.581, 0.408, 0.559, 0.126, 1.000, 0.128, 0.416, 0.441, 0.235, 0.191, 0.131, 0.128, 1.000, 0.058, 0.257, 0.183, 0.029, 0.382, 0.416, 0.058, 1.000, 0.381, 0.714, 0.231, 0.581, 0.441, 0.257, 0.381, 1.000, 0.448, 0.531, 0.408, 0.235, 0.183, 0.714, 0.448, 1.000, 0.271, 0.559, 0.191, 0.029, 0.231, 0.531, 0.271, 1.000 ), nrow = 7, byrow = TRUE)

print(true_cov_matrix)

true.theta2 <- rmvnorm(N, mean = mu, sigma = true_cov_matrix)

mod2222 <- generate.mirt_object(parameters, itemtype = 'graded', latent_covariance = true_cov_matrix)

responsepattern2 <- simdata(mo = mod2222, Theta = true.theta2)

CAT simulation

real_sim_Trule <- mirtCAT(mo = mod2222, local_pattern = responsepattern2, method = 'MAP', criteria = "Trule", design = design2)

real_simitemsanswered <- laply(real_sim_Trule, function(x) length(x$items_answered)) mean(real_simitemsanswered) ``

[philchalmers]

Your example isn't reproducible. What is item.param, where does the nearPD() come from, etc. Please ensure that the reprex is a standalone example.

[Hsin21]

I'm sorry for providing the unreproducible code earlier. Attached are text files of my data and code. I appreciate your help again. Thank you.

If there are any questions or missing information, please feel free to let me know. I apologize if the way I provided the code was inappropriate, as I'm not very familiar with R yet. I hope this time it runs smoothly. Thank you once again.

example.txt new3.xlsx

[philchalmers]

The code you provided still isn't a reprex. Try using source('example.txt') in your R session and you'll see that there are errors. I'm closing this issue for now, but if you can come up with a reprex demonstrating the issue (and why you think it's an issue theoretically) then feel free to reopen.