search

kenkellner / jagsUI

R package to Run JAGS (Just Another Gibbs Sampler) analyses from within R
https://kenkellner.github.io/jagsUI/
34 stars 8 forks source link

Calculating WAIC in jagsUI #19

Closed cwliu007 closed 9 months ago

cwliu007 commented 6 years ago

Martyn Plummer added an example to calculate WAIC at https://sourceforge.net/p/mcmc-jags/discussion/610036/thread/8211df61/. Is it the same way to do in jagsUI? Thanks!

kenkellner commented 6 years ago

I suspect that if you have JAGS 4.3 (maybe it has to be a dev version?) installed and add 'WAIC' to your list of parameters to save (and set DIC=T), jagsUI will handle this properly like any other monitored parameter. Based on that link it seems the final form of the WAIC monitor is still in flux. I will keep an eye on it and when it's final figure out how to handle it in jagsUI. I certainly would like to include it.

kenkellner commented 6 years ago

I think we're waiting for the next version of JAGS (>4.3.0) for this built-in WAIC functionality, I'll check again when it's released.

orbidder commented 5 years ago

Just to add, I've been monitoring the log-likelihood at each iteration by calculating it in the main loop. For my logistic regression model that means using log.like[i]<- logdensity.bin(y[i] , regression_prob[i] , 1). There isn't always documentation available for those logdensity functions though. But anyway, once you monitor log.like, you can use them as input to the loo package waic() function. Calculating loo is a little more tricky, as you have to use the chain_id argument in relative_eff() to map the matrix that I think jagsui outputs.

GrahamBirch commented 3 years ago

Hi Ken,

I'm trying to calculate WAIC modelling with JAGSUI too. Mines a state transition model and I can't work out how to apply the example given by the OP with the dcat distribution i've used to specify Y.

Anyway, does JAGS just calculate WAIC for you like DIC? Did you add this to JAGSUI? I tried adding "WAIC" as a monitored parameter but if it is there I can't see it...?

kenkellner commented 3 years ago

As far as I know, JAGS 4.3 is still the most recent version available, and jagsUI doesn't have built-in WAIC support. You might consider using Nimble instead, specifically the nimbleMCMC function. It has an argument for calculating WAIC (I've not used it myself) and you should be able to use your existing BUGS code without any changes.

GrahamBirch commented 3 years ago

Wow thank for getting back to me so quickly! Is there a reason JAGS hasn't been updated in so long? It seems to me it doesn't have a lot of recent active discussion in general online so don't know if it's just fallen out of favour?

kenkellner commented 3 years ago

I don't know the exact reason, but JAGS is developed primarily by one person who surely has many other responsibilities. Also, there are more similar tools available now (Stan, Nimble, Greta, etc.) compared to even just a few years ago, so JAGS doesn't get as much focused attention.