Closed qbouniot closed 7 months ago
All modified and coverable lines are covered by tests :white_check_mark:
Project coverage is 99.04%. Comparing base (
740ae5b
) to head (1d0988b
). Report is 10 commits behind head on dev.
:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.
Thanks, @qbouniot, for the PR!
@alafage, shouldn't we implement AdaptiveCE as a special case of CE? Just add a new parameter adaptive that would default to False?
Btw, do we add a num_calibration_bins
param to the classification routine?
Edit: after discussion with @alafage we settled for yes to both questions.
This is a proposition of implementation for the Adaptive ECE metric into
torch_uncertainty/metrics/classification/calibration.py
. I've also added it by default in the multiclass classification routine. There is also a minor fix when doing temperature scaling with the test set. Since the test datasets are returned as a list, (ID test set + OOD test set), the dataset used will be the first element of the list by default.TO DO:
Would fix #83.