search

vturrisi / solo-learn

solo-learn: a library of self-supervised methods for visual representation learning powered by Pytorch Lightning
MIT License
1.39k stars 181 forks source link

The results of SimCLR is not match with the original paper #305

Closed HuangChiEn closed 1 year ago

HuangChiEn commented 1 year ago

At the first, thansk for your release of such amazing framework, which almost cover all of the SOTA-SSL methods.

While enabling the multi-GPU training (up to 4 gpus), the linear evaluation results do not match the SimCLR report in Table B.1 in appendixes. image

By the way, I have rearranged the config file by easy-configer, so that it's easy to execute in basah and easy read in wandb.

The results in wandb based on solo-learn framework can be found in the following link : https://wandb.ai/josef/solo-learn/runs/ah1hwzpm/overview In short, we found that we got 60.83% (about 4% drop) of linear-eval Top-1 acc.

Any suggestion will be appreciated!!

vturrisi commented 1 year ago

Hi. Thanks for using our library. I would first check if all parameters match those in the SimCLR paper. Apart from that, I'm not sure if we have any issues with SimCLR, but I can eventually take a closer took at it.

vturrisi commented 1 year ago

Hi. I took a brief look at your config, and your eta for Lars is wrong. The default values for Lars should parameters be:

kwargs:     clip_lr: False     eta: 0.001     exclude_bias_n_norm: True

They differ from those for Imaneget100 as we found that a larger eta benefits the models there. There might be other issues that I didn't spot as well.

HuangChiEn commented 1 year ago

Hi. I took a brief look at your config, and your eta for Lars is wrong. The default values for Lars should parameters be:

kwargs:     clip_lr: False     eta: 0.001     exclude_bias_n_norm: True

They differ from those for Imaneget100 as we found that a larger eta benefits the models there. There might be other issues that I didn't spot as well.

Sorry, reply lately and thanks for your comment. Yes, i think i will adjust the eta values according to your suggestion.