oracle / macest

Model Agnostic Confidence Estimator (MACEST) - A Python library for calibrating Machine Learning models' confidence scores
Universal Permissive License v1.0
100 stars 20 forks source link

feat: ability to use only precomputed point predictions #4

Open FlorentRamb opened 2 years ago

FlorentRamb commented 2 years ago

This PR adds the ability to train and use a macest model only from precomputed point predictions. This allows to have no reference to the point predcition model in the macest model, which makes the loading/saving and usage of these models easier. The following changes are introduced:

Signed-off-by: Florent Rambaud flo.rambaud@gmail.com

rgreen1995 commented 2 years ago

Hi, @FlorentRamb great to see you working on this and improving the code! just to check how this works, is this allowing you to do a similar pre-computation in prediction as we do when training the model? i.e. when training we calculate the point predictions once at the beginning and then pass these predictions around when training all the mace parameters. So is this MR allowing you to do this after training?

FlorentRamb commented 2 years ago

If I'm understanding well, yes indeed! The optional parameter prec_point_preds is added to ModelWithConfidence.predict_confidence_of_point_prediction and ModelWithPredictionInterval.predict_interval. Using it allows you to pre-compute the point predictions of your ml model and then feed them to the macest model even at prediction time.