Doubt about zero risk predictions and the zero threshold

[giulianonetto]

Hello, Daniel!

First of all, thanks for writing dcurves and making it publicly available!

Please, I have a more conceptual question if you don't mind.

My understanding is that dcurves subtracts a very small value from risk predictions that equal exactly 0 (commit). So, when a model originally predicts zero risk, or when a binary test is negative, you get "negative classifications" in the net benefit calculation even if the decision threshold is zero. Is my understanding correct? My intuition for DCA was that, for the zero threshold, any model or binary test should yield a "positive classification", regardless of the original risk prediction of the test result. I feel like I am missing something.

I should disclose that the reason I'm so interested is that I am writing a (Bayesian) DCA package myself (manuscript in preparation) and am unsure how to handle zero thresholds properly.

Thank you for any help!

Best, Giuliano

[ddsjoberg]

Depending on how one defines a true positive, you'll see something strange at either thresholds of 0 or 1. If you want to keep things simple, just don't allow users to supply these thresholds.

Alternatively, you can think of a threshold of 0 as only being the same of a treat all scenario

[giulianonetto]

I see!

I was going for treating a threshold of 0 as the same as the Treat All strategy, but it ended up yielding weird-looking DCAs for binary tests (example below).

I will probably avoid confusion and not allow a threshold of 0, as you suggested.

Thank you for your help!