search

EFS-OpenSource / calibration-framework

The net:cal calibration framework is a Python 3 library for measuring and mitigating miscalibration of uncertainty estimates, e.g., by a neural network.
https://efs-opensource.github.io/calibration-framework/
Apache License 2.0
336 stars 42 forks source link

Question about input range of multivariate confidence calibration #51

Closed junhyung-SPALab closed 9 months ago

junhyung-SPALab commented 9 months ago

Hello, I would like to ask a question that arose while doing research using the great platform you provide. This question is about the function fit() of the class Abstract Calibration implemented in netcal/AbstractCalibration.py. Looking at line 164, regardless of the task (whether classification or detection), the range of input X is limited to a value between 0 and 1.

If calibration is performed using box parameters together, elements such as width and length will be outside the range. Is there a reason why you implemented it as above? Also, if I want to use box parameters, could you please recommend how to convert them to that range and calibrate them?

fabiankueppers commented 9 months ago

Hi @junhyung-SPALab, the main reason we restrict the valid input range to the [0, 1] interval for all cases was that it is consistent across all possible tasks on the one hand. On the other hand, if you think of the BetaCalibration or BetaCalibrationDependent methods, due to the underlying Beta distributions, their input range is restricted to [0, 1] by design.

I recommend normalizing your box parameters by the image height/width (at least we did it like that) to obtain bbox parameters in a [0, 1] range.

I hope this helps.