stan-dev / rstanarm

rstanarm R package for Bayesian applied regression modeling
https://mc-stan.org/rstanarm
GNU General Public License v3.0
388 stars 133 forks source link

Allow kfold to optionally return the cross-validated stanreg-objects #161

Closed paasim closed 7 years ago

paasim commented 7 years ago

Summary:

Currently the kfold function in rstanarm performs K-fold cross-validation, but only returns the elpd-values (and some other summaries). While this is clearly the primary target of the function, allowing the function to return also the cross-validated stanreg-objects would make this function usable also for the glmproj-package (and possibly other packages interfacing rstanarm).

Description:

The glmproj-package uses K-fold cross-validation to produce summary statistics for submodel size selection. The current kfold function in the rstanarm package would be perfect if it would also return the cross-validated stanreg-objects. (We essentially need to extract the posterior draws of the weight vector and the dispersion parameters as well as the indices of the omitted observations for each of the K folds.)

This could be achieved for example by adding an argument save_fits, which would default to FALSE (as most of the time the user probably does not need the stanreg-objects). Setting it to TRUE would just save the stanreg-objects and the indices of the omitted observations to an array in the output of the function.

I added a propossal of the new version of the function (the only differences to the current function are lines 17, 31 and 42 which I added). This should already work and is suitable for our needs, so if this seems ok, I can do a pull request for this, but I am also open for any comments. (Or if you think this is something that should not be in rstanarm, we can do this in glmproj as well.)

kfold_new.txt

jgabry commented 7 years ago

Thanks @paasim. I'll take a look. I think I'm ok with adding this extra argument, especially if it will make glmproj easier to use with rstanarm models

jgabry commented 7 years ago

Ok I think having the option to save the fits is fine. @paasim Want to make a pull request for this?

jgabry commented 7 years ago

closed via #163