check your models! - Githubissues

lizzieinvancouver / ospree

Budbreak review paper database

3 stars 0 forks source link

check your models! #285

Closed lizzieinvancouver closed 5 years ago

lizzieinvancouver commented 5 years ago

please report here:

you lowest n_eff (and report here if most were not close to max iter) ...
check the methods in supp regarding the models correspond to your models! Adjust supp as needed (or explain here)
confirm your Rhat values @dbuona weinberger @cchambe12 latitude @AileneKane other models

lizzieinvancouver commented 5 years ago

@AileneKane @dbuona @cchambe12 We also need everyone to be using the same versions of R and Stan ...

dbuona commented 5 years ago

For weinberger- lowest n_eff was 3217.162 all rhats are 0.999.. or 1.0

I'm running: R version 3.5.2 (2018-12-20) -- "Eggshell Igloo" and stan version "2.18.0"

AileneKane commented 5 years ago

@dbuona @cchambe12 @lizzieinvancouver I have just updated R to the latest version and am updating all the main budburst model iterations from table 2S (utah, cp, centered, uncentered, all spp). If we can all use the latest version, i think that would be ideal. R version 3.6.0 (2019-04-26)

cchambe12 commented 5 years ago

Using version 3.6.0 Lowest n_eff was 713.4 (but most were close to max) Rhats are 0.999 to 1.00

dbuona commented 5 years ago

reran weinberger model on 3.6.0 Rhats are all 1.00... lowest neff was 2857 ...I'll get to work on a table for this model

lizzieinvancouver commented 5 years ago

@cchambe12 @dbuona Can you all check that you had 1 500 warm-up iterations followed by 2 500 sampling iterations? This should yield 10 000 samples. @cchambe12 I especially wanted you to double check this and the 713 n_eff (as that would be below 10% ... but close). Thanks!

AileneKane commented 5 years ago

@lizzieinvancouver @cchambe12 @dbuona I believe that our models are actually coded to have 2500 total iterations (1500 are used for warm-up). This should yield 4000 samples. I wrote the text incorrectly in our manuscript. Sorry for the confusions!

cchambe12 commented 5 years ago

Yes that sounds right, thanks Ailene! And yes, my code has 1,500 warm-up iterations and 2,500 sampling iterations.

dbuona commented 5 years ago

mine too.

lizzieinvancouver commented 5 years ago

@AileneKane @cchambe12 @dbuona Ah! Thanks everyone for checking and sorry for my panic. This means are n_eff are very good! Nice work!

lizzieinvancouver commented 5 years ago

@AileneKane I think if we correct the supp text to make it clear there were 2500 total iterations 1500 are used for warm-up ... we should be good. We have good n_eff and Rhats so the models have converged on just 4K posterior samples, pretty good.

And maybe we can close this soon?