ip200 / venn-abers

Python implementation of binary and multi-class Venn-ABERS calibration
MIT License
114 stars 11 forks source link

Multiclass implemetation of pre-fitted classifiers #16

Closed ip200 closed 4 months ago

ip200 commented 4 months ago

In standard IVAP and CVAP multi-class calibration, we consider a collection of binary classification problems and then combine their solutions to obtain multi-class probabilities. For some classifiers (e.g. Deep Neural Networks) it may not be practical to modify the algoritthm to generate a set of one-vs-one probability outputs. In this case, we can use the calibration set multi-class probabilities to calibrate the outputs in a similar way, by converting them into equivalent one-vs-one outputs first for all binary class pair combinations in the calibrations set and applying the VennABERS procedure directly to them.