scikit-learn-contrib / MAPIE

A scikit-learn-compatible module to estimate prediction intervals and control risks based on conformal predictions.
https://mapie.readthedocs.io/en/latest/
BSD 3-Clause "New" or "Revised" License
1.27k stars 102 forks source link

Allow `MapieRegressor` to use any split strategy #386

Closed thibaultcordier closed 9 months ago

thibaultcordier commented 9 months ago

Description

Split-CP is only available with "split" keyword or ShuffleSplit class of scikit-learn.

This PR proposes to use more compatible split BaseCrossValidator/BaseShuffleSplit.

In practice, we need to use the predictions aggregation for cross-CPs and not for split-CPs. All you need to do is find out the number of splits using the get_n_splits method to know whether it's a split or a cross.

Fixes #385

Type of change

How Has This Been Tested?

Please describe the tests that you ran to verify your changes. Provide instructions so we can reproduce. Please also list any relevant details for your test configuration

Checklist

thibaultcordier commented 9 months ago

This PR also tidies things up a bit: the tests specific to CQR have been moved to the dedicated file (from test_utils.py to test_quantile_regression.py).

codecov-commenter commented 9 months ago

Codecov Report

All modified and coverable lines are covered by tests :white_check_mark:

Comparison is base (614293e) 100.00% compared to head (1fa8b7c) 100.00%. Report is 34 commits behind head on master.

Additional details and impacted files ```diff @@ Coverage Diff @@ ## master #386 +/- ## ========================================== Coverage 100.00% 100.00% ========================================== Files 39 39 Lines 4616 4633 +17 Branches 487 762 +275 ========================================== + Hits 4616 4633 +17 ```

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.