Accessing p0p1

[tuvelofstrom]

I think it would be valuable if there was an easy way of accessing p0p1 as an optional additional output when calling predict_proba.

In the application we are using VennAbers for, we need both the calibrated probability and the probability interval [low, high] (or p0p1). We have solved it with an additional parameter which is false by default. See predict_proba at row 60 in calibrated_explanations/VennAbers.

[tuvelofstrom]

After some thought, I realized that I could use the venn_abers.VennAbers class directly. Consequently, none of the issues I initiated are strictly needed on my account. However, I believe both issues are worth considering for other use cases as well, so think about whether they could be worthwhile implementing regardless!

[ip200]

Hi Tuwe, thank you so much for your suggestions! I will aim to incorporate these this weekend and will drop you a line as soon as its done. Speak shortly, Ivan

[ip200]

Hi Cristian

Yes, absolutely, you can access p0 and p1 values by calling the VennAbers.predict_proba() function directly. It returns calibrated probabilities and the associated p0 and p1 values. Please see https://github.com/ip200/venn-abers/blob/main/src/venn_abers.py line 220 onwards. Additionally, you can set the parameter p0_p1_output=True in VennAbersCalibrator ( https://github.com/ip200/venn-abers/blob/main/src/venn_abers.py, line 860) but please note this works for binary classification problems only.

I hope this is OK.

Thanks,

Ivan

On Tue, 16 Jan 2024 at 19:35, Cristian Del Toro @.***> wrote:

Hello. I'd like to know, Is this possible already?

— Reply to this email directly, view it on GitHub https://github.com/ip200/venn-abers/issues/2#issuecomment-1894388867, or unsubscribe https://github.com/notifications/unsubscribe-auth/AJJSY4GKEMQDMPTKT3J6RVDYO3I6VAVCNFSM6AAAAAA4YFCFAWVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTQOJUGM4DQOBWG4 . You are receiving this because you commented.Message ID: @.***>

[cmyguel]

Hello Ivan, Thank you for your previous response.

In my use case, I require probability intervals p0p1 for a multiclass application with 3 classes.

Is there a straightforward method to obtain these probabilities without an in-depth understanding of the library? If not, would it be acceptable to train 3 models in a one-vs-rest fashion to acquire these values? Being very rigorous is not a primary concern; I need practical and usable predictions for my application.

[ip200]

Hi Cristian

Thank you for your message and sorry for the delay in replying. At the moment po_p1 probabilities are only available for binary classification problems. However I think there can be a relatively straightforward way of adding functionality to the library so that p0_p1 outputs are generated as one-vs all for the most probable class output. So for example, for. a 3-class problem with classes A, B, C if the output for a given example is predicted to be C, p0 would give the probability that the output is not C and p1 that it is. This could be easily extended to any n-class problem. Would that be useful?

Thanks,

Ivan

On Wed, 24 Jan 2024 at 19:51, Cristian Del Toro @.***> wrote:

Hello Ivan, Thank you for your previous response.

In my use case, I require probability intervals p0p1 for a multiclass application with 3 classes.

Is there a straightforward method to obtain these probabilities without an in-depth understanding of the library? If not, would it be acceptable to train 3 models in a one-vs-rest fashion to acquire these values? Precision is not my primary concern; I need practical and usable predictions for my application.

— Reply to this email directly, view it on GitHub https://github.com/ip200/venn-abers/issues/2#issuecomment-1908814185, or unsubscribe https://github.com/notifications/unsubscribe-auth/AJJSY4AVY2IDUNHCIRINB2TYQFQ35AVCNFSM6AAAAAA4YFCFAWVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTSMBYHAYTIMJYGU . You are receiving this because you commented.Message ID: @.***>