Thie1e / cutpointr

Optimal cutpoints in R: determining and validating optimal cutpoints in binary classification
https://cran.r-project.org/package=cutpointr
85 stars 13 forks source link

How to access ppv values given a custom cutpoint #55

Closed jwang-lilly closed 2 years ago

jwang-lilly commented 2 years ago

Hi Christian,

Using the following code, I can get to opt_cut for a custom cutpoint. How do I access the PPV and NPV values from the 100 samplings? I'd like to plot the PPV or NPV distribution. I figure it has something to do with opt_cut$boot but I am not able to figure it out. Thanks much.

` library(cutpointr) library(dplyr)

Optimal cutpoint for dsi

data(suicide) opt_cut <- cutpointr(data = suicide, x = dsi, class = suicide, pos_class = 'yes', neg_class = 'no', direction = '>=', boot_stratify = TRUE, method = oc_manual, cutpoint = 3, boot_runs = 100) %>% cutpointr::add_metric(list(ppv, npv)) ` @Thie1e

Thie1e commented 2 years ago

Hi Jian,

as a starting point, this will plot the distribution of out-of-bag values of PPV:

library(cutpointr)
library(tidyverse)

data(suicide)
opt_cut <- cutpointr(data = suicide, 
                     x = dsi, 
                     class = suicide,
                     pos_class = 'yes', 
                     neg_class = 'no',
                     direction = '>=', 
                     boot_stratify = TRUE,
                     method = oc_manual, 
                     cutpoint = 3, 
                     boot_runs = 100) %>%
    cutpointr::add_metric(list(ppv, npv))
#> Running bootstrap...

res_boot_unnested <- opt_cut %>% 
    select(boot) %>% 
    unnest(boot) %>%
    select(-roc_curve_b, -roc_curve_oob)

# '_oob' indicates out-of-bag values, '_b' indicates in-bag values
ggplot(res_boot_unnested, 
       aes(x = ppv_oob)) + 
    geom_density() + 
    ggtitle("Bootstrap", "out-of-bag values of PPV with manual cutpoint = 3") + 
    geom_rug(alpha = 0.5)

Created on 2022-02-26 by the reprex package (v2.0.1)

jwang-lilly commented 2 years ago

This is perfect! Thanks so much.