search

arviz-devs / arviz

Exploratory analysis of Bayesian models with Python
https://python.arviz.org
Apache License 2.0
1.61k stars 403 forks source link

Rhat documentation is not clear #1462

Open jmaronas opened 3 years ago

jmaronas commented 3 years ago

Hi all.

I have been going through the Arviz library. In particular, when looking at the Rhat I see the following in the documentation here https://arviz-devs.github.io/arviz/api/generated/arviz.rhat.html

I see five options given but they don't specify which actual Rhat is computing. I can more or less infer it but I think the documentation could be improved at this point. The options are:

Select R-hat method. Valid methods are: - “rank” # recommended by Vehtari et al. (2019) - “split” - “folded” - “z_scale” - “identity”

In my opinion and from my understanding what I infer is that the options provided correspond to:

identity: Standard Rhat proposed in Gelman 1992 split: Split Rhat in the Bayesian Data Analysis Book rank: split rhat over rank values z_scale: equation 14 in Vehtari et al 2019 folded: equation 15 in Vehtari et al 2019.

I think this could be improved and explained a bit more. On the other hand, I dont realy agree with “rank” # recommended by Vehtari et al. (2019). From my understanding Vehtari et al 2019 recommends a combination of z_scale and folded, see section 4.2 from Vehtari et al 2019.

Please, let me know if I have misinterpreted the documentation at this point. In any case I think it could be improved.

jmaronas commented 3 years ago

I have been going through the code and I think I know how arviz is doing things under the hood. I think the documentation can be improved for easy understand. I would write something like:

identity: Original Rhat as proposed in Gelman 1992 split : Split Rhat as proposed in Gelman 2013 z_scale : Rank normalized split Rhat as proposed in vehtari et al section 4.1 equation 14 folded : Rank normalized split Rhat over folded values as proposed in vehtari et al section 4.2: equation 15 + 14 rank : Summary Rhat recommended by Vehtari et al section 4.2; max(z_scale,folded)

aloctavodia commented 3 years ago

Hi @jmaronas , thanks for your interest. Do you want to do a PR?

jmaronas commented 3 years ago

sure, I'll do it. Which branch?

aloctavodia commented 3 years ago

Create a new feature branch. More details here