search

frankkramer-lab / aucmedi

a framework for Automated Classification of Medical Images
https://frankkramer-lab.github.io/aucmedi/
GNU General Public License v3.0
38 stars 13 forks source link

Discrepancy between AUC score and ROC plot #165

Closed SherlockMones closed 2 years ago

SherlockMones commented 2 years ago

Testing landmark classification, that results in bad AUC scores (between 0.4 and 0.6), although ROC shows impeccable classification. F1 also amazing scores (~ 0.9). See screeenshots for details

image

image

muellerdo commented 2 years ago

https://scikit-learn.org/stable/modules/generated/sklearn.metrics.roc_auc_score.html

muellerdo commented 2 years ago

Before: plot performance barplot

After: plot performance barplot

muellerdo commented 2 years ago

Issue was incorrect probability access if no threshold is provided (automatic). For some logic error I thought I had to pick the maximum softmax of all classes which is obviously incorrect...

Fixed it! :tada:

muellerdo commented 2 years ago

PR started

muellerdo commented 2 years ago

Merged in dev branch.