search

stan-dev / loo

loo R package for approximate leave-one-out cross-validation (LOO-CV) and Pareto smoothed importance sampling (PSIS)
https://mc-stan.org/loo
Other
150 stars 34 forks source link

accelerate the optimization algorithm in stacking #119

Open yao-yl opened 5 years ago

yao-yl commented 5 years ago

stacking_weights() optimize the weights, which is a (K-1) simplex. Currently, it renders the constrained optimization and optimizes the first (K-1) weights in a constraint space. When the number of models is large, it is slow (e.g., more than 10 minutes for 10000 models * 100 data)

I did some experiment and found it would be much quicker if it transformed the simplex in the unconstrained space first (several seconds for the same task above). The easiest way is to call optimizing() in rstan and it does the transformation automatically. It also makes it easier if we need extra prior on weights.

@jgabry What is a good way to execute a pre-compiled stan object, without making the whole package dependent on rstan?

jgabry commented 4 years ago

That's a good idea but I don't know how to do it without rstan (or having R call out to cmdstan). @bgoodri would know better than I would, but I don't see how loo could execute a pre-compiled stan program from R without having to do something similar to what the rstanarm package does.