henrikbostrom / crepes

Python package for conformal prediction
BSD 3-Clause "New" or "Revised" License
449 stars 35 forks source link

Documentation #14

Closed karllandheer closed 1 year ago

karllandheer commented 1 year ago

Hello, thank you very much for your great package. I have a quick question, where does the equation for calculate_p come from? It seems to be roughly just comparing the input predicted probabilities (i.e., softmax values) to the rank-ordering of the calibration non-conformity scores. This kind of makes sense, but does this really correspond to a p-value necessarily? Is there a reference for this?

henrikbostrom commented 1 year ago

You find in-depth background in:

Vovk, V., Gammerman, A. and Shafer, G., 2005. Algorithmic learning in a random world. Springer

Here is a recent introduction:

Angelopoulos, A. N., & Bates, S. (2023). Conformal prediction: A gentle introduction. Foundations and Trends® in Machine Learning, 16(4), 494-591.

Best regards, Henrik

henrikbostrom commented 1 year ago

This issue is moved to "Discussions".