EFS-OpenSource / calibration-framework

The net:cal calibration framework is a Python 3 library for measuring and mitigating miscalibration of uncertainty estimates, e.g., by a neural network.
https://efs-opensource.github.io/calibration-framework/
Apache License 2.0
344 stars 42 forks source link

netcal.binning.BBQ.transform() sometimes returns values that are outside of the [0,1] range #54

Closed Praisethefab closed 3 months ago

Praisethefab commented 8 months ago

Code to reproduce issue: `

insert here any model to calculate the confidence array, I got this error with multiple different models in multiple different datasets for binary classification

            bbq_calibration = BBQ()
            bbq_calibration.fit(y_conf_cal[:,1], y_cal)
            y_conf_bbq = bbq_calibration.transform(y_conf_cal[:,1])

` sometimes the y_conf_bbq would contain values that go outside 0 and 1, I suspect that it is a floating point error since when I tested to see what numbers it gave outside the [0,1] range I got 1.0000000000000002, but as it was relatively rare I did not try multiple times to see wether different anomalous values are possible. If indeed it is a floating point error simply clipping the output should be fine to fix this error.

fabiankueppers commented 3 months ago

Hi @Praisethefab, thank you very much for raising attention on this issue. I will add clipping within the next release of netcal. Thanks!