mikevoets / jama16-retina-replication

JAMA 2016; 316(22) Replication Study
https://doi.org/10.1371/journal.pone.0217541
MIT License
110 stars 37 forks source link

About replication details #1

Closed socoi closed 6 years ago

socoi commented 6 years ago

Hi. I am also researching on DR topic. I focus the Google work also. I notice that AUC for Messidor dataset reached around 0.59 in your work. I have trained all images from Kaggle( total 88702) without excluding any images and make a binary classification. And test my model on Messidor dataset. It reaches about 0.94 AUC score.

I use Resnet50 as the backbone of the network. Because some lesions like red small dots are extremely small, I set the size of the input image as 448*448. I refer some preprocessing procedures from first prize team from Kaggle. Many papers do not use EyePacs for testing. I suppose you may try using the data for training and validate only.

But my model achieved quite a poor performance on another dataset DIARETDB1. Would you mind testing your model on this dataset? Besides, have you ever tried detecting lesions in fundus images?

mikevoets commented 6 years ago

The main goal of this work was to replicate Google's JAMA paper, which we refer to in the readme of this repo. We didn't necessarily focus on getting the best performance. Instead, we wanted to know if we could get similar results by following their methodology as accurately as possible.

I just skimmed through the DIARETDB1 data set. However as far as I can see it does not include any direct referable DR grades, it includes ground truths of lesions locations instead. If there aren't direct rDR grades I will need some time to "translate" these lesions ground truths into rDR labels.

I am also working on my "own" approach that doesn't follow Google's approach, but it is more leaning towards approaches from other Kaggle competitors, with several data augmentation techniques and for example larger image sizes. That work is soon to be released in another repo. So stay tuned :-)

mikevoets commented 6 years ago

Our research team is working on a second edition of the article with some new results after trying to improve the model, apart from replication. When it's done we will update the readme of this repo with a link to the article. Stay tuned! I'll close this issue for now.

wenwenyu commented 6 years ago

@socoi Hi. I am working on DR topic recently too, but my results is not good. Can you mind to share your project about this? Thinks

socoi commented 6 years ago

@WENWENYU I am still testing my model. If you like to know some details, we can have a free talk in Chinese through e-mail.

wenwenyu commented 6 years ago

@socoi OK,thinks!