search

stan-dev / loo

loo R package for approximate leave-one-out cross-validation (LOO-CV) and Pareto smoothed importance sampling (PSIS)
https://mc-stan.org/loo
Other
148 stars 34 forks source link

Port C++ psis code to loo? #260

Open SteveBronder opened 3 months ago

SteveBronder commented 3 months ago

In Stan we have a lot of the code for psis sampling available as C++.

https://github.com/stan-dev/stan/blob/develop/src/stan/services/pathfinder/psis.hpp

It should be a good bit faster. How much time do you ballpark the psis sampling takes / would it be a lot better if it was faster in R?

We could do this by writing some Rcpp to call the functions in Stan. However this would be a large dependency to add so if the current speed is fine then idt we should worry about this too much.

But if speed is an issue happy to put this on my plate to do at some point

jgabry commented 4 days ago

Thanks @SteveBronder, just saw this (finally catching up on GitHub issues!). Yeah I think the only reason not to do this would be adding the dependency on Rcpp. I guess, like you said, it all depends on whether the added speed would be a game changer. PSIS smoothing is definitely the biggest bottleneck when running loo(), but I've never tried comparing the R and C++ versions to see how much of a speedup there would be. @avehtari what do you think?