scikit-learn-contrib / MAPIE

A scikit-learn-compatible module to estimate prediction intervals and control risks based on conformal predictions.
https://mapie.readthedocs.io/en/latest/
BSD 3-Clause "New" or "Revised" License
1.2k stars 99 forks source link

Added regression_MWI_score function #408

Closed Carl-McBride-Ellis closed 4 months ago

Carl-McBride-Ellis commented 5 months ago

Description

This is an efficient implementation for calculating the mean Winkler interval score, based on the code written by Kaggle user AmbrosM and published in the topic "Accelerating the scoring function".

Fixes #407

Type of change

How Has This Been Tested?

Private testing was performed in the Kaglge competition Prediction interval competition I: Birth weight by comparing correspondence with the results to those output by the competition metric code published in Winkler Interval score metric.

Update (2024-02-27)

A test routine for the function regression_MWI_score has been written and added to the end of mapie/tests/test_metrics.py (b101cb0).

Checklist

codecov-commenter commented 4 months ago

Codecov Report

All modified and coverable lines are covered by tests :white_check_mark:

Project coverage is 100.00%. Comparing base (614293e) to head (5ebeee1). Report is 267 commits behind head on master.

Additional details and impacted files ```diff @@ Coverage Diff @@ ## master #408 +/- ## ========================================== Coverage 100.00% 100.00% ========================================== Files 39 39 Lines 4616 4896 +280 Branches 487 801 +314 ========================================== + Hits 4616 4896 +280 ```

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

LacombeLouis commented 4 months ago

We should definitely add some documentation, but we can do this in a separate PR, so this is pushed through as quickly as possible!

New issue: #415.