search

lightly-ai / lightly

A python library for self-supervised learning on images.
https://docs.lightly.ai/self-supervised-learning/
MIT License
2.92k stars 248 forks source link

Be more clear about the models and mention the difference in implementation compared to the papers #93

Open IgorSusmelj opened 3 years ago

IgorSusmelj commented 3 years ago

For SimCLR and MoCo there are two versions. Our implementations are more close to v2 of both from what I've seen. But I'm not sure we use all the changes from the papers. I think it makes sense to clarify which model lightly is using and if there is a difference to the paper we should mention this. Maybe also mention how one could use SimCLRv1 or SimCLRv2 by changing some of the parameters?

philippmwirth commented 3 years ago

@IgorSusmelj can you elaborate on the exact differences? If we have them in a TODO list format we can easily start working on them.

IgorSusmelj commented 3 years ago

This is still open. The point is, that for some implementations it is unclear whether we followed the papers. And some papers have follow-up papers. E.g. SimCLR has two (v1: https://arxiv.org/abs/2002.05709, v2: https://arxiv.org/abs/2006.10029). Moco also has two (v1: https://arxiv.org/abs/1911.05722, v2: https://arxiv.org/abs/2003.04297) The goal of the issue is to figure out which version we follow for the two of them. It's a great issue to also learn more about the two models. I highly recommend everyone interested in self-supervised learning have a look at SimCLR and MoCo.

pranavsinghps1 commented 3 years ago

I will get it done!

pranavsinghps1 commented 3 years ago

From what I could figure out both - Moco and SimCLR existing in the codebase are V1.