Path not found for calibrator model when using "sybil_base"

[maxZ90]

Nice project. "sybil_ensemble" works fine, however, there seems to be a problem with the localization of the calibrator to "sybil_base".

I am also a little bit confused by the calibrator concept. I noticed your comment about sklearn.calibration.CalibratedClassifierCV, but i was wondering howto create a calibrator when training sybil with my own data. I would be happy if you could explain the role and creation of a calibrator in more detail.

[pgmikhael]

Hi,

Thank you for pointing this out! We recently changed how the model files are automatically downloaded, so using sybil_1 instead of sybil_base should work.

Regarding the calibrator, deep learning models are known to produce predictions that are not well-calibrated. To obtain calibrated predictions (closer to true class probabilities), we fix the model and simply take the predictions to learn a separate model that calibrates those scores (could be as simple as learning a single parameter). More details about this process in general: https://scikit-learn.org/stable/modules/calibration.html.

Hope this helps!