search

jeremykawahara / derm7pt

Dermatology dataset composed of a diagnosis and seven-point checklist criteria labels
Other
69 stars 14 forks source link

AUCROC in paper #2

Open Oktai15 opened 5 years ago

Oktai15 commented 5 years ago

Hi, @jeremykawahara, @hamarneh!

Can you explain how we calculate AUCROC for diagnosis? AUCROC is for binary classification, how can we interpret this for multi-class classification (because 1 diagnosis per image)?

Moreover, the same question about AUCROC in every criteria.

Do you use just AUCROC from sklearn that can return dict with AUCROC per label and use AUCROC like binary classification to every label?

It will be very cool, if you explain this point or publish code that calculate metrics from paper.

jeremykawahara commented 5 years ago

Hello @Oktai15, the AUROC score was computed with respect to each class label where all other class labels are considered as the negative class. For example, when we report the AUROC score for the melanoma diagnosis label, melanoma is considered the positive class and all other diagnosis labels are considered negative. So yes we are considering this as a binary labelling.

I hope to release more code soon and will let you know if/when we do so. Thanks!