scikit-learn-contrib / MAPIE

A scikit-learn-compatible module to estimate prediction intervals and control risks based on conformal predictions.

https://mapie.readthedocs.io/en/latest/

BSD 3-Clause "New" or "Revised" License

1.2k stars 99 forks source link

FIX bug #452: Quantile Correction #453

Closed thibaultcordier closed 3 weeks ago

thibaultcordier commented 1 month ago

Description

Use the corrected quantile ($|(n+1)(1−α)|/n$) to check the validity of the coverage for any number of calibration data.

Fixes #452

Type of change

Bug fix (non-breaking change which fixes an issue)

How Has This Been Tested?

[x] Perform a FWER controlling procedure (Bonferroni) to ensure the coverage control 1−α < coverage < 1−α + 1/(n+1).
[in next PR] Reproduce the experiment of Mahdi Torabi Rad (see his notebook: https://github.com/mtorabirad/MLBoost/blob/main/Episode15/Episode15Main.ipynb) in the special case where the number of calibration data is very small (here 6).

Checklist

[x] I have read the contributing guidelines
[x] I have updated the HISTORY.rst and AUTHORS.rst files
[x] Linting passes successfully : make lint
[x] Typing passes successfully : make type-check
[x] Unit tests pass successfully : make tests
[x] Coverage is 100% : make coverage
[x] Documentation builds successfully : make doc