search

blackjax-devs / sampling-book

Tutorials and sampling algorithm comparisons
https://blackjax-devs.github.io/sampling-book
68 stars 13 forks source link

Roll our own summary #6

Closed rlouf closed 1 year ago

rlouf commented 1 year ago

Numpyro has a nice numpyro.diagnostics.print_summary function to print diagnostics for the chains. However:

  1. It feels wasteful to import numpyro just for this functionality;
  2. Blackjax implements the same diagnostics
  3. Blackjax may have more diagnostics in the future, see https://github.com/blackjax-devs/blackjax/issues/384 and https://github.com/blackjax-devs/blackjax/issues/278

I think we should thus implement our own function that prints a summary with different diagnostics; this will certainly be useful when comparing the results from different algorithms.

junpenglao commented 1 year ago

I think we should use https://python.arviz.org/en/stable/ for this.

Note that we have rhat and ess currently for the reason that we might want to explore using these for tuning. So I will apply the same logic when consider adding new diagnostics)

albcab commented 1 year ago

Agree with @junpenglao that we don't need to be a diagnostics library, just use arviz.

But maybe we can use some of these diagnostics on testing?

rlouf commented 1 year ago

I agree with both of you, closing for now.