search

alexander-rakhlin / Return-to-Diabetic-Retinopathy-Detection

New project, powerful model
Other
16 stars 5 forks source link

Labels for Messidor-2 #3

Closed mikevoets closed 6 years ago

mikevoets commented 6 years ago

In your paper you mention that you used the Messidor-2 set for testing. As far as I know the Messidor-2 set does not come with DR labels, in contrast to Messidor-Original. Did you assess DR for the images in Messidor-2 yourself, or did you have an ophthalmologist who labeled the images for your research?

alexander-rakhlin commented 6 years ago

It is here: https://medicine.uiowa.edu/eye/abramoff

mikevoets commented 6 years ago

Thanks! Strange that they are not uploaded here http://latim.univ-brest.fr/indexfce0.html.

In your study you used a definition of referable diabetic retinopathy (rDR) of ICDR levels 1 to 4. As far as I can see, is that the labels on that website (Messidor-2 labels) assess rDR defined by ICDR levels 2 to 4. How were you able to test Messidor-2 for your definition of rDR?

alexander-rakhlin commented 6 years ago

They aren't same indeed, and I pointed this out in the paper. My model might be more sensitive to level 1 DR than others reported on Messidor-2, but I can not attest this qualitatively.

mikevoets commented 6 years ago

OK, understood. What you could do is get the Messidor-Original data set and use the labels from there to evaluate. They didn't grade by using ICDR, but since your model detects mild DR as rDR you can simply translate the Messidor-Original labels to 0 and 1. And then use the image pairing labels in Messidor-Original which can be retrieved from Messidor-2 for improved performance as well.

Thanks for your insights :)