SimCLR example

[sdesrozis]

🚀 Feature

Idea is to provide new class of example for self supervised learning.

It could be an implementation of paper "A Simple Framework for Contrastive Learning of Visual Representations" https://arxiv.org/abs/2002.05709

Things to do are

• normalized temperature-scaled cross entropy loss (NT-Xent loss)
• Pair-transformed dataset
• CIFAR example using resnet
• compare to imagnet pretraining
• use idist for distributed training

[vfdev-5]

This also may be a good FR for ignite.data module.

[vfdev-5]

Maybe, it would be better to reimplement BYOL too : https://arxiv.org/abs/2006.07733

[sdesrozis]

Yes, it's not so far from SimCLR. I will do it in my research lab during next few months.