search

aakashrkaku / intermdiate_layer_matter_ssl

The official repository for "Intermediate Layers Matter in Momentum Contrastive Self Supervised Learning" paper.
40 stars 7 forks source link

Many issues with the code / reproducability #2

Closed nlml closed 2 years ago

nlml commented 2 years ago

Hello,

I am trying to reproduce some of the results in your paper.

I am focused for now on the Diabetic Retinopathy results, and in particular the file ssl_mocov2_mse.py

It seems there are many bugs in this code, including syntax errors and the use or passing of undefined variables, such that it doesn't run.

Just one example - in your main function here: https://github.com/aakashrkaku/intermdiate_layer_matter_ssl/blob/main/diabetic_retinopathy/ssl_mocov2_mse.py#L521-L544

You define dm = BaselineDataModule ... but then you are looking for a variable called data_module

I can try to fix the errors like this (and also make a Dockerised version of the training to make reproducability easier). But if you could provide a script that actually runs and produces your results, that would be very useful!

Thanks a lot, and thanks for sharing your research code!

aakashrkaku commented 2 years ago

Hi Liam,

We apologize that you faced difficulty in reusing our code. We have updated our code and ran some tests to ensure it works without errors. But, if you still find some errors, please do let us know. We will resolve the issues as soon as possible.

Thank you, Aakash

nlml commented 2 years ago

Thank you! I will let you know.

I was able to fix the errors without too much effort and the results still seemed good. In fact (and I still have to double check), but it seemed like I was getting results that were quite a bit better even (around .89 ROC on test set for linear evaluation of features).