Closed itsdfish closed 3 years ago
Thanks Chris, will try to switch to your setup for these tests.
The crucial line in compute_loo() is where I multiply the log_prob with the log_weights ( lwp += psis_output.weights
).
Chris, works fine. Any chance you could do the cars example in Turing. If not, I will attempt it early next week.
Your example gives me a loo of 190.
Carlos, you probably understand this much better than I do, but both PSIS and Waic are to compare models, so a constant offset in the values might be less important?
I'm currently adding helper functions to compute LOO values, along with MCSE and other related quantities provided by the R loo package; these will go in another package, since some uses of PSIS are not related to LOO-CV (e.g. improving variational approximations).
Hi @ParadaCarleton and @goedman
@ParadaCarleton, thanks again for putting together this package.
Given that I have very limited knowledge of PSIS and LOO, I may not be the best person for this task. Nonetheless, based on what I can piece together, what appears to be missing are helper functions for computing pointwise log likelihoods and the loo value from the psis object. Here is my best attempt. Am I on the right track?