arviz-devs / arviz-stats

Statistical computation and diagnostics for ArviZ.
https://arviz-stats.readthedocs.io
Apache License 2.0
2 stars 1 forks source link

Add prior sense related functionality #24

Closed aloctavodia closed 1 month ago

aloctavodia commented 2 months ago

Closes #23

From the user perspective the main addition in this PR is the psens_summary function, which is equivalent to powerscale_sensitivity from the priorsense package.

azs.psens_summary(sc)

     prior  likelihood  diagnosis
mu      0.413   0.584   prior-data conflict
sigma   0.297   0.538   prior-data conflict

Where sc is a DataTree from a PyMC model equivalent to univariate_normal model in priorsense docs. Results are close to those, differences may be do to sampling differences, I need to check more carefully. There are also small differences compared to the result from az.psens (now the samples are the same).


📚 Documentation preview 📚: https://arviz-stats--24.org.readthedocs.build/en/24/

OriolAbril commented 1 month ago

I have only looked at the psens function, not the psens_summary one. I think everything works as expected. Hopefully what is going on is somewhat clear. I think results shoud exactly match current arviz now, there was an issue in passing down the dimensions on which to operate dims->axes so instead of axes being defined from dims it was set to the default -1 (instead of the correct (-2, -1) when using data with chain, draw).

Now both:

import arviz_stats
from arviz_base import load_arviz_data, extract
idata = load_arviz_data("non_centered_eight")
arviz_stats.psens(idata, group="log_likelihood")

and

arviz_stats.psens(
    idata.drop_nodes(["prior", "prior_predictive", "observed_data", "constant_data"]).stack(sample=["chain", "draw"]),
    group="log_likelihood",
    sample_dims="sample"
)

work and return the same result.