search

deepskies / DeepDiagnostics

Inference diagnostics for mostly SBI
MIT License
1 stars 0 forks source link

New Metric: Area Under Coverage #36

Open bnord opened 9 months ago

bnord commented 8 months ago

integrate the area under coverage curve to produce a single number that represents the relative expected confidence of the model.

bnord commented 7 months ago

@voetberg is this equivalent to the coverage fraction in https://github.com/deepskies/DeepDiagnostics/blob/main/src/metrics/coverage_fraction.py?

voetberg commented 7 months ago

I think that's a question for @beckynevin , what I wrote there is just a straight adaptation of her coverage_fraction code

bnord commented 6 months ago

@beckynevin what does the coverage_fraction code calculate?