arviz-devs / arviz-plots

ArviZ modular plotting
https://arviz-plots.readthedocs.io
Apache License 2.0
3 stars 2 forks source link

Add plot_compare #77

Closed aloctavodia closed 2 months ago

aloctavodia commented 3 months ago

closes #76

There are two major points I want to discuss for this PR (or we can split the discussion into more than one PR).

about the second point I would like to simplify the plot and only show the ELPD point estimates + SE. Then we will get rid of the arguments insample_dev, plot_standard_error, plot_ic_diff, order_by_rank, legend. The main reason for this is that these are the most useful elements for the plot and this is how most people use and interpret this plot (including the equivalent plot in the Stan world).

I also would like to add two new arguments (already in the docstring), that I think could help make the plot easier to interpret.

Captura desde 2024-08-02 11-23-08

Update

This is how it looks like now in

matplotlib output_matplotlib

plotly (the band is missing from plotly, not sure why yet) output_plotly

bokeh output_bokeh

codecov-commenter commented 3 months ago

Codecov Report

Attention: Patch coverage is 85.29412% with 10 lines in your changes missing coverage. Please review.

Project coverage is 84.71%. Comparing base (60aac9a) to head (6742f7a). Report is 2 commits behind head on main.

Files with missing lines Patch % Lines
src/arviz_plots/plots/compareplot.py 90.32% 6 Missing :warning:
src/arviz_plots/plot_collection.py 0.00% 4 Missing :warning:
Additional details and impacted files ```diff @@ Coverage Diff @@ ## main #77 +/- ## ========================================== + Coverage 83.84% 84.71% +0.86% ========================================== Files 20 21 +1 Lines 2229 2336 +107 ========================================== + Hits 1869 1979 +110 + Misses 360 357 -3 ```

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

OriolAbril commented 3 months ago

I really like the proposed changes. Completely agree on similar_band=True as default, not sure about relative, it is true we don't care about the absolute value, but I expect people to worry about the best one being exactly 0. We could try having it on and if we see people reporting back and getting confused we can turn it off