Open yao-yl opened 3 years ago
Yeah it would be nice to support this. Regarding the extra dependence on Stan, would the loo package have to run the Stan code from step 3 for the user?
Perhaps in this case the loo package can serves as a stan-model-generator: compute pointwise loo, suggest a default stacking model, and generate a stan file. Then user can run/modify this generated stan file on their own.
Is it possible that there could be dependence on rstan, but such dependence is only checked/brought when certain loo::function is called?
Perhaps in this case the loo package can serves as a stan-model-generator: compute pointwise loo, suggest a default stacking model, and generate a stan file. Then user can run/modify this generated stan file on their own.
Interesting idea. Would be curious to know what other people think of this option, but yeah that's a possibility.
Is it possible that there could be dependence on rstan, but such dependence is only checked/brought when certain loo::function is called?
In theory yes, but RStan isn't a typical package so we'd need to check if that works (but I think it should). Do you need to call rstan::log_prob
or rstan::grad_log_prob
? If not then I think it would actually be easier to do this with CmdStanR than RStan.
Given the new research progress on "hierarchical stacking", it would be useful to implement this new tool into the loo package.
This new tool needs more user customization: after a few models are fit, the user needs to
The challenge for putting this package into the present package lies in that