search

GenoML / genoml2

GenoML (genoml2) is an open source Python package. It is an automated machine learning (autoML) platform for genomics data
Apache License 2.0
27 stars 17 forks source link

ROC not reporting same value as exported predictions #9

Closed m-makarious closed 4 years ago

m-makarious commented 4 years ago

Please make sure that this is a bug.

System information

Describe the current behavior: @h-leonard found an issue with RandomForestClassifier reporting an AUC in the performance metrics file that is not consistent with the ROC figure produced

Describe the expected behavior: ROC reported should match what is output with performance metrics (seems to only be this algorithm... others seem to be working fine)

Code to reproduce the issue: Please provide a reproducible test case that is the bare minimum necessary to generate the problem.

I was able to replicate this issue by commenting out the competing algorithms and just report RandomForestClassifier AUC was reported at 50 and ROC reports 48

EXAMPLE image

Other information or logs: Please include any logs or source code that would be helpful to diagnose the problem. If including tracebacks, please include the full traceback. Large logs and files should be attached and not pasted, for clarity

Not entirely sure where to start or why this is the case...

m-makarious commented 4 years ago

Addressed this issue in the most recent commits, reported values should now be consistent with what is reported in the ROC figure regardless of algorithm that performs best

If there are additional issues with this, please let me know!

m-makarious commented 4 years ago

Moved issue to new repo for completeness and consistency