search

p-lambda / verified_calibration

Calibration library and code for the paper: Verified Uncertainty Calibration. Ananya Kumar, Percy Liang, Tengyu Ma. NeurIPS 2019 (Spotlight).
MIT License
141 stars 20 forks source link

Question about num_calibration #1

Closed TortoiseHam closed 3 years ago

TortoiseHam commented 3 years ago

Hi, thanks for making this library - it's super convenient. One question about the PlattBinnerMarginalCalibrator (and I suppose the other calibrators as well). What is the purpose of the num_calibration argument? it seems like it's only being used to assert that a minimum number of samples have been provided, but what is the point of that? Does it have any usages apart from the single assert?

AnanyaKumar commented 3 years ago

Yup that's the only usage - probably would've been better to not have it! Thanks for the question, and for pointing this out.

TortoiseHam commented 3 years ago

Cool, thanks for confirming