Getting three differing errors in fit domain, which I do not understand

[WrdeVries]

Hi,

I have tried the fit domain function on several of the tests in my dataset. I seem to be getting three different errors, of which I am unsure what they mean and or how to resolve them. If needed I can attach a minimal example.

> fit_domains(db,item_property = "questiontype", booklet_id =="10048")
Error in solve.default(H * scale, E) : 
  Lapack routine dgesv: system is exactly singular: U[1,1] = 0
In addition: Warning message:
In fit_inter_(.) :
  every score can be reached in only one way, no data reduction possible

> fit_domains(db,item_property = "questiontype", booklet_id =="10039")
The following items have no zero score category:
[1] "choice"
Error in fit_inter_(.) : data contains items without zero score category

> fit_domains(db,item_property = "questiontype", booklet_id =="10012")
Error in solve.default(H * scale, E) : 
  system is computationally singular: reciprocal condition number = 1.9513e-17

[jessekps]

Hi,

The fit_domains function is nice in principle but in practice it puts some demands on your dataset which are difficult to exactly define and in these cases cause the errors.

Fit_domains works by summing the scores of the tiems in a particular domain, so that the domain becomes the item. If the items are fairly easy, or there are many items in a domain, it can easily occur that nobody gets 0 points on a domain. The way the models in dexter are set up means that there has to be a zero score for each item, otherwise it won't calibrate. This is what causes the second error above, which admittedly could have been more clear.

The first error is caused by something similar, only now it apears that one of the new 'items' (=domains) has quite many missing score categories and possibly one of the new items is very much bigger than the others so particular test scores can only be reached by means of getting a single possible score on one of the items,which leads to an unidentified model and an non-invertible matrix of second derivatives, which causes the first error.

The third error probably has a similar casue as the first. What the technical explanation boils down to practically is that your data is probably not suitable for summing by "questiontype" because the scores get too extreme or sparse in some way. It is possible that this can be resolved by gathering more responses but without looking at your data I really cannot say.

[WrdeVries]

Thank you for this clearly laid out answer. This answers my questions. It has also helped me decide that, for now, I will not be using the fit domain function.

I do wonder whether Dexter has other options that could help me analyse two differing domains. These domains being: multiple choice and constructed responses (or open ended text questions). Preferably in a way that I can compare between tests.

[jessekps]

There are some options.

• I often use profile_plot to compare mc and open-ended questions across paper and digital administrations. But they require a division across a person as well as an item variable (and are per booklet).
• If you're worried whether different types of items measure the same skill you can use a latent correlation latent_cor
• To compare different types of items in the absence of a relevant person distinction, my most common strategy is to use classical statistics averaged over booklets and compare the distributions. See the below example. You can do this over booklets assuming the itemtypes are somewhat equally distributed over booklets and he populations for the different booklets are not very different.

library(dexter)
library(dplyr)
library(ggplot2)

db = start_new_project(verbAggrRules, ":memory:")
add_booklet(db, verbAggrData, "agg")
add_item_properties(db, verbAggrProperties)

tia=tia_tables(db,type='averaged')

items = get_items(db) |> inner_join(tia$items,by='item_id')

ggplot(items,aes(x=rir,color=mode,fill=mode)) +
  geom_density(alpha=0.5) +
  theme_minimal() +
  xlim(0,1)

ggplot(items,aes(x=pvalue,color=mode,fill=mode)) +
  geom_density(alpha=0.5) +
  theme_minimal() +
  xlim(0,1)

[WrdeVries]

Thank you for the response. I will check out whether the latent_cor yields interesting results in my case.

For now, I am using the profile_tables function to calculate expected domain scores. I can then use these in a further analysis which is based on a paper by N. Verhelst : Balance: A Neglected Aspect of Reporting Test Results. If you are interested I can elaborate further.

Thanks a lot for the explanations!

[jessekps]

That's another good direction. All depends on what your purpose is I suppose. I know the article. If something interesting comes out I'd be interested to hear about it. You can email me, the address is in the description file. Good luck!