There is an issue with using calibration on some metrics such as precision, where the results at the requested thresholds are not shown (only the 'optimal' result is being shown).
This is the expected result when using calibration with f1 (working).
calibration = sim_model.calibrate(
x=x,
y=y,
calibration_metric='f1',
extra_metrics=['precision', 'recall'],
thresholds_targets={f"{v/10}": v/10 for v in range(10)},
)
The bug when using calibration with precision - only the optimal result is shown, even though higher precision is requested in calibrate:
calibration = sim_model.calibrate(
x=x,
y=y,
calibration_metric='precision',
extra_metrics=['f1', 'recall'],
thresholds_targets={f"{v/10}": v/10 for v in range(10)},
)
There is an issue with using calibration on some metrics such as precision, where the results at the requested thresholds are not shown (only the 'optimal' result is being shown).
This is the expected result when using calibration with f1 (working).
The bug when using calibration with precision - only the optimal result is shown, even though higher precision is requested in calibrate: