scikit-learn-contrib / MAPIE

A scikit-learn-compatible module to estimate prediction intervals and control risks based on conformal predictions.

https://mapie.readthedocs.io/en/latest/

BSD 3-Clause "New" or "Revised" License

1.2k stars 99 forks source link

ENH: externalize MapieClassifier methods into several non-conformity scores #476

Open thibaultcordier opened 3 days ago

thibaultcordier commented 3 days ago

Description

Outsource the MapieClassifier methods (LAC, APS, RAPS, TOPK) by extending the non-conformity score class to the classification task.

Fixes #467

Type of change

New feature (non-breaking change which adds functionality)
Breaking change (fix or feature that would cause existing functionality to not work as expected)
This change requires a documentation update

How Has This Been Tested?

[x] Test that existing code continues to work
[ ] Test that the proposed solution can enable new non-conformity scores

Checklist

[x] I have read the contributing guidelines
[x] I have updated the HISTORY.rst and AUTHORS.rst files
[x] Linting passes successfully : make lint
[x] Typing passes successfully : make type-check
[x] Unit tests pass successfully : make tests
[x] Coverage is 100% : make coverage
[x] Documentation builds successfully : make doc

vincentblot28 commented 1 day ago

What a tour de force @thibaultcordier !! Great improvement to the classification API ! General comment about the example: as in the future we prefer to use directly the conformity scores (and the argument "method" might be removed) I think examples should follow the new way of using MapieClassifier