search

scikit-learn-contrib / MAPIE

A scikit-learn-compatible module to estimate prediction intervals and control risks based on conformal predictions.
https://mapie.readthedocs.io/en/latest/
BSD 3-Clause "New" or "Revised" License
1.27k stars 102 forks source link

Support for LGBM Booster models #403

Open caeduft opened 8 months ago

caeduft commented 8 months ago

Conformal prediction in MAPIE using existing models (e.g. https://mapie.readthedocs.io/en/latest/examples_regression/1-quickstart/plot_prefit.html#sphx-glr-examples-regression-1-quickstart-plot-prefit-py) supports models which have fit and predict attributes only. However, LGBM Regression models saved to disk, either using Booster method or pickle, don't have a "fit" attribute but a "refit" attribute (https://lightgbm.readthedocs.io/en/latest/pythonapi/lightgbm.Booster.html#lightgbm.Booster.refit). Unfortunately, MAPIE seems unable to use the refit method. Therefore, at this point, an LGBM Booster instance seems to be incompatible with MAPIE.

Enabling compatibility between LGBM Booster and MAPIE could be very useful for instances where a model is reused multiple times for prediction in the future and fitting a model from scratch is not possible.

LacombeLouis commented 8 months ago

Hey @caeduft, thank you for your message. So indeed, this is something that could be fixed using a wrapper and naming the refit method fit. Please checkout issue #340 where we give an example of this. Note that you will only be able to use a split setting.

This is indeed something we want to fix more generally by giving more examples of this in the documentation, so do not hesitate to share your code afterwards!