tidymodels / probably

Tools for post-processing class probability estimates
https://probably.tidymodels.org/
Other
115 stars 15 forks source link

Use average of predicted probabilities instead of bin midpoint #112

Open agormp opened 1 year ago

agormp commented 1 year ago

Feature

This package looks extremely useful. Pretty plots, and very useful with the rugs and the options for windowing!

One suggestion: Currently the value on the x-axis is the bin midpoint. I would suggest to instead use the average of the predicted probabilities included in that bin: this will be the expected value of the event rate.

(For instance, if a bin only has predicted probabilities near the lower edge of the bin range, then we would also expect an event rate near that lower edge. Using the average predicted probability will sort this out)

This will mostly matter when using few bins.