Open strengejacke opened 5 years ago
All the *_performance functions computes in-sample based measures. From my point of view the word "performance" suggests how well the model works out-of-sample. Would it be possible to add out-of-sample measures like RMSE, MAE, out-of-sample R^2, and for classification: AUC, precision, recall +++
I have formulas for all of the SEM fit indices under the sun. I can take that.
Regarding out of sample stuff, I would also love if performance could do various true cross-validation stuff, both individual residuals and aggregates like RMSE, R^2^, and deviance. Those are currently strewn across various packages.
Regarding PCA/factor analysis, to meet most user expectations, that's going to require at least a handful of dependencies like GPArotation. Various nuances of rotations and extraction methods are pretty fiddly, but also already well-implemented in psych and fungible. I would suggest these functions should be wrappers, probably starting with the princomp, factanal, and psych::fa() and psych::fa.parallel() functions.
Providing a unifying syntax for extracting, interpreting and understanding their models could be quite appreciated. Although this will probably wait for the help of a future contributor, expert in this kind of things.
Isn't that essentially the aim of tidymodels?
This would definitely fit.
Other things that come to my mind when I think of the scope:
Originally posted by @DominiqueMakowski in https://github.com/easystats/performance/issues/14#issuecomment-469009121