Adaptative / Conditional Conformal Prediction

[thibaultcordier]

Is your feature request related to a problem? Please describe.

Despite attractive theoretical guarantees and practical successes, Predictive Interval (PI) given by Conformal Prediction (CP) may not reflect the uncertainty of a given model. This limitation arises from CP methods using a constant correction for all test points, disregarding their individual uncertainties, to ensure coverage properties. (From Salim I. Amoukou, N. Brunel (2023)).

Different methods are proposed to deal with this problem, which we have grouped under the name "adaptive conformal prediction" (see additional context).

Describe the solution you'd like

The objective of this enhancement is to propose conditional / adaptative conformal prediction for MapieRegressor and MapieClassifier (in two different issues/PRs ?).

Describe alternatives you've considered

One way to achieve this is to adapt the fit and predict methods to the different classes of ConformityScore.

Additional context

In "adaptive conformal prediction", we distinguish different approaches (non exhaustive list):

• Reweighting of the nonconformity score
  • Localized Conformal Prediction (LCP): Guan, L. (2021). Localized Conformal Prediction: A Generalized Inference Framework for Conformal Prediction. Biometrika.
  • SLCP: Han, X., Tang, Z., Ghosh, J., & Liu, Q. (2022). Split Localized Conformal Prediction. ArXiv, abs/2206.13092.
  • LCP-RF: Amoukou, S.I., & Brunel, N.J. (2023). Adaptive Conformal Prediction by Reweighting Nonconformity Score. ArXiv, abs/2303.12695.
• Renormalization of the nonconformity score
  • Locally adaptive split conformal prediction (also known as conformalized residual fitting - CRF): Papadopoulos, H., Gammerman, A., & Vovk, V. (2008). Normalized nonconformity measures for regression Conformal Prediction.
  • Self-supervised Conformal Prediction (SSCP): Seedat, N., Jeffares, A., Imrie, F., & Schaar, M.V. (2023). Improving Adaptive Conformal Prediction Using Self-Supervised Learning. ArXiv, abs/2302.12238.
• Estimation of the nonconformity score
  • Conformalized Quantile Regression (CQR): Romano, Y., Patterson, E., & Candès, E.J. (2019). Conformalized Quantile Regression. Neural Information Processing Systems.

[jesuinovieira]

It seems that this issue was closed together with #305 🤔

[thibaultcordier]

Good point, thank you for this comment :)

[joshdunnlime]

What about Self-supervised Conformal Prediction (SSCP)? Is that likely to get added? +1 from me :)