search

eddatasci / unrollment_proj

The Unrollment Project: Exploring algorithmic bias in predicting bachelor's degree completion.
5 stars 0 forks source link

Accuracy function #21

Open wdoyle42 opened 4 years ago

wdoyle42 commented 4 years ago

Function that will take results of model (probabilities, classification) and generate an accuracy measure for the resampled data.

Inputs: model results Outputs: AUC across all resamples

btskinner commented 4 years ago

If I understand correctly, @wdoyle42, your intention is to wrap

## get AUC across bootstraps
auc <- results %>%
    filter(!is.na(.pred_Completed)) %>% 
    group_by(id) %>%
    roc_auc(truth = truth,
            prediction = .pred_Completed)

in a more general function along the lines of

get_auc <- function(<results df>, <true value>, <predicted value>)

where <results> are a data frame in the general form returned by grad_class() that includes at least two columns: the true value of the outcome and a predicted value of the outcome.

Is this correct?

wdoyle42 commented 4 years ago

@btskinner, yes, that's right. The roc_auc obviously does most of this, the question is whether we want to build a function that also generates graphics, or if we want to subset or reformulate the output from roc_auc in any way.

mpatricia01 commented 4 years ago

Hi @btskinner & @wdoyle42 ! Is it okay with you both if I spend some time learning how to use github some more before taking a stab at this issue? I am a little rusty at the moment and never took the time to learn the ins and outs of github-- mostly the basics.

Would also appreciate any resources you recommend.

btskinner commented 4 years ago

@mpatricia01, that's totally fine with me. If you haven't seen it, Jenny Bryan has great resource for using git with R: https://happygitwithr.com. Always feel free to send any questions my way, too.

@wdoyle42, I'll take this issue over.