search

jasp-stats / jasp-issues

This repository is solely meant for reporting of bugs, feature requests and other issues in JASP.
55 stars 29 forks source link

Mixed Models: assumption checks and model performance #836

Open TarandeepKang opened 4 years ago

TarandeepKang commented 4 years ago

Is your feature request related to a problem? Please describe.

**Describe the solution you'd like** At the moment, the mixed modelling module is certainly substantial, but as far as I’m aware it does not allow the entire analysis pipeline to be completed in JASP. I suggest some additions based upon my reading of: https://doi.org/10.1016/j.jml.2020.104092 and https://doi.org/10.7717/peerj.4794 First of all, it is suggested that Variance Inflation Factors should be used to help assess multicollinearity in predictors, with predictors having a value created greater than three being removed. https://doi.org/10.1111/j.2041-210X.2009.00001.x This is implemented in the vif.lmer function from the PiecewiseSEM package. This and a variety of other checks of assumptions and overall model performance can be calculated using the performance package: https://easystats.github.io/performance/ It is certainly advisable to try to include as many of the performance checks in the software as possible, and possibly also add the ability to compare the model fit of null and experimental models hierarchically. Thank you all for your great work. It is not currently possible to perform any of these checks using software with a graphical user interface, therefore they can only be done in R. With the addition of some/all these changes, the user will be able to report that all assumptions of mixed models are met and provide a performance summary, which will therefore bring In the line with the latest best practice guidelines for the use of mixed models.
EJWagenmakers commented 4 years ago

@FBartos (maybe contact Henrik about this as well)

TarandeepKang commented 2 years ago

Hi Team, I'm just wondering if there has been any progress with this request?

EJWagenmakers commented 2 years ago

I do like these ideas, particularly since we are now also going to implement a more careful check on the residuals. @fqixiang @FBartos @Kucharssim (seems relevant to all of you)

TarandeepKang commented 1 year ago

Dear team, I'm just writing to check if there's been any updates with regard to my request? And, perhaps more interestingly, to provide a brief update to new developments regarding a framework of R squared measures for mixed models, and the necessary R package:

Rights, J. D., & Sterba, S. K. (2019). Quantifying explained variance in multilevel models: An integrative framework for defining R-squared measures. Psychological Methods, 24, 309–338. https://doi.org/10.1037/met0000184

Rights, J. D., & Sterba, S. K. (2020). New Recommendations on the Use of R-Squared Differences in Multilevel Model Comparisons. Multivariate Behavioral Research, 55(4), 568–599. https://doi.org/10.1080/00273171.2019.1660605

Rights, J. D., & Sterba, S. K. (2023). R-squared Measures for Multilevel Models with Three or More Levels. Multivariate Behavioral Research, 58(2), 340–367. https://doi.org/10.1080/00273171.2021.1985948

Shaw, M., Rights, J. D., Sterba, S. S., & Flake, J. K. (2023). r2mlm: An R package calculating R-squared measures for multilevel models. Behavior Research Methods, 55(4), 1942–1964. https://doi.org/10.3758/s13428-022-01841-4

confidence intervals for R2s generated with r2mlm can be computed with:

Lai, M. H. C. (2021). Bootstrap Confidence Intervals for Multilevel Standardized Effect Size. Multivariate Behavioral Research, 56(4), 558–578. https://doi.org/10.1080/00273171.2020.1746902

TarandeepKang commented 8 months ago

Another new development in LMM model performance/ selection:

Säfken, B., Rügamer, D., Kneib, T., Greven, S., 2021. Conditional Model Selection in Mixed-Effects Models with cAIC4. Journal of Statistical Software 99, 1–30. https://doi.org/10.18637/jss.v099.i08

TarandeepKang commented 4 months ago

Hi team, I was just wondering if there is a chance I could give this one a nudge?