DistrictDataLabs / yellowbrick

Visual analysis and diagnostic tools to facilitate machine learning model selection.
http://www.scikit-yb.org/
Apache License 2.0
4.3k stars 559 forks source link

Draw ROC curve #1225

Closed whire97612 closed 2 years ago

whire97612 commented 2 years ago

As we all know, drawing ROC curve requires FPR and TPR data. How can I use yellowbrick when I export FPR and TPR data using deep learning model?thanks!

lwgray commented 2 years ago

@whire97612 Thank you so very much for the question. We have a couple of options, both are apart of our contrib visualizer which are not primetime but have limited functionality. The first option is the use your deep learning model with Yellowbrick. This is demonstrated here. https://www.scikit-yb.org/en/stable/api/contrib/wrapper.html

The second option is to use our Prepredict Visualizer which would allow you to just specify your y_pred and y_test. However, I tested it with roc_auc and it didn't work because roc_auc requires a predict_proba method. The docs are here: https://www.scikit-yb.org/en/stable/api/contrib/prepredict.html

You can try these by upgrading to the latest version of yellowbick by pip install -U yellowbrick

@bbengfort Do you have anything else to add?

bbengfort commented 2 years ago

@lwgray I second your suggestions for the wrapper and prepredict module - I did look at the ROCAUC source code, and unfortunately tpr and for are computed inside the visualizer. It may be possible to decouple it but that would require at least one new visualizer. @whire97612 let us know if one of these wrappers works for your model!