[Feature Request]: dynamic fit indices for CFA

[TarandeepKang]

%### Description

Fit indices for CFA that take into account sample characteristics and can be considered an improvement upon widely used benchmarks such as Hu and Bentler

Purpose

We know that fit indices are influenced by the characteristics of the data being analysed and so previously established cut-offs may not be generalisable

Use-case

Whenever we wish to get a more accurate picture of the fit of a CFA

Is your feature request related to a problem?

Mounting research shows that previously established cut-offs may not always be reliable/ valid

Is your feature request related to a JASP module?

Factor

Describe the solution you would like

Implement dynamic fit indicices within JASP

Describe alternatives that you have considered

Continue using existing cut-offs despite shortcomings, or conduct CFA in R

Additional context

Dynamic fit indices are recommended as alternatives to just taking previously established interpretation guidelines at face value. These approaches often outperform other commonly used methods such as equivalence testing or the default cut-offs. And have recently been implemented in the "dynamic" R package. This works on an existing lavaan model object which is already produced by the CFA procedure in JASP.

McNeish, D. (2023). Generalizability of Dynamic Fit Index, Equivalence Testing, and Hu & Bentler Cutoffs for Evaluating Fit in Factor Analysis. Multivariate Behavioral Research, 0(0), 1–25. https://doi.org/10.1080/00273171.2022.2163477 McNeish, D., & Wolf, M. G. (2021). Dynamic fit index cutoffs for confirmatory factor analysis models. Psychological Methods. https://doi.org/10.1037/met0000425 McNeish, D., & Wolf, M. G. (2022). Dynamic fit index cutoffs for one-factor models. Behavior Research Methods. https://doi.org/10.3758/s13428-022-01847-y Wolf, M. G., & McNeish, D. (2023). dynamic: An R Package for Deriving Dynamic Fit Index Cutoffs for Factor Analysis. Multivariate Behavioral Research, 0(0), 1–6. https://doi.org/10.1080/00273171.2022.2163476

[juliuspfadt]

@TarandeepKang, thank you for the request and for providing all the info about the fit index cutoffs.

I have to say I am a bit torn on the topic. So far, we provide the fit indices without any guidance on the cutoffs, since this is quite the contentious topic, and I think users should educate themselves on cutoffs and if they want to apply those. In general, I do not like the accept/reject philosophy of the cutoff-game. Now the dynamic cutoffs do improve on the status quo as they adjust to characteristics of the model. Nevertheless, they keep the accept/reject logic, and they shift the responsibility of accepting a model or not to the software instead of the user as it is likely that the fit indices will be used at face value instead of careful consideration. Of course, one can argue that so far, people use fit indices at face value anyways based on Hu and Bentler, but at least they are responsible for that instead of being tempted by JASP. Any thoughts @Kucharssim, @LSLindeloo, @EJWagenmakers.

[TarandeepKang]

I am torn, too. I would suggest that most people are only interpreting values in line with prespecified cut-off criteria. And we basically know those cut-off criteria are not all they are cracked up to be. So yes, well you might hope that people will make informed decisions, not everybody is a stats/methodology specialist like us. So, people are making informed decisions on the cut-off criteria they don't need these dynamic indices. But the majority who probably aren't, definitely do. Providing more, and better options, I would argue is never a bad thing. I entirely agree is not obvious what one should do. I am absolutely not saying that these dynamic indices must be implemented, I just think it might be beneficial. I don't have a fixed position, but I'm glad I seem to have started a discussion. You are very welcome, Julius!

[EJWagenmakers]

We could offer these upon request, and provide some context so that they are used responsibly

[TarandeepKang]

Exactly, the kind of output that might be more difficult/unnecessary for some users, and so might only be provided to people who explicitly want it. You seem to do this in a number of places across the software already.

Also, just to bring this discussion bang up-to-date on literature, McNeish recently published this additional work which is very relevant to our discussion.

McNeish, D. (2023). Dynamic fit index cutoffs for categorical factor analysis with Likert-type, ordinal, or binary responses. The American Psychologist, 78(9), 1061–1075. https://doi.org/10.1037/amp0001213

McNeish, D., & Manapat, P. D. (2024). Dynamic Fit Index Cutoffs for Hierarchical and Second-Order Factor Models. Structural Equation Modeling: A Multidisciplinary Journal, 31(1), 27–47. https://doi.org/10.1080/10705511.2023.2225132

This approach has also now been generalised to other models including SEM McNeish, D., & Wolf, M. G. (2023). Direct Discrepancy Dynamic Fit Index Cutoffs for Arbitrary Covariance Structure Models. PsyArXiv. https://doi.org/10.31234/osf.io/4r9fq

McNeish, D. (2023). Dynamic Fit Index Cutoffs for Treating Likert Items as Continuous. https://osf.io/8bt2m/