search

deepskies / DeepDiagnostics

Inference diagnostics for mostly SBI
MIT License
1 stars 0 forks source link

Parity plots #29

Closed beckynevin closed 3 months ago

beckynevin commented 6 months ago

This could be expanded to parity difference and parity difference covariance plots.

bnord commented 6 months ago

for modularity, should these each be their own function (parity diff, parity difference covariance, parity), or should they inherit(wc?) from each other?

bnord commented 6 months ago

Parity plots = 1-1 plots = p-p plots = true vs. predicted plots

bnord commented 6 months ago

parity difference plot = true - predicted vs. true (y-axis vs x-axis)

variants include residual: (true - predicted / true) vs. true percentage: (true - predicted / true)*100 vs. true

bnord commented 6 months ago

often the parity difference plot goes directly underneath the parity plot and share their x-axes.

bnord commented 6 months ago

The parity difference covariance plot is a corner plot, where they independent and dependent variables are the difference between true and expected. JasonPoh came up with this, and has examples in his strong lensing repository.

bnord commented 3 months ago

Here are some examples of parity plots

bnord commented 3 months ago

1to1plot_Ana

screenshot_537
bnord commented 3 months ago

these are residuals like what's in the lower panel of the above comment. 1_param_corner_42.pdf

bnord commented 3 months ago

This is an example of a parity difference plot 5param_corner_465.pdf

voetberg commented 3 months ago

image Any strong feelings on this? The difference, residuals, and percentage plots are all optional.

bnord commented 3 months ago

I like this. I also envision a few variations. But, I think we can go with this as the baseline, and then we can make some updates later. Does that sound right?