Loss function and optimizer

sthalles / SimCLR

PyTorch implementation of SimCLR: A Simple Framework for Contrastive Learning of Visual Representations

https://sthalles.github.io/simple-self-supervised-learning/

MIT License

2.19k stars 457 forks source link

Loss function and optimizer #20

Closed SiminLi closed 3 years ago

SiminLi commented 3 years ago

Hi Thalles, I went through the code and find two things I can't understand: (1) in code "labels = torch.zeros(2 * self.batch_size).to(self.device).long()", in nt_xent.py, seems the label is constantly 1. So the label is not used since its all 0? (2) is adam + "scheduler = torch.optim.lr_scheduler.CosineAnnealingLR" same as the LARS optimizer?

Thanks in advance.

sthalles commented 3 years ago

Hello guys, that is a general message to say that I have refactored the whole project. Please I a look at the new impl and free to submit PR if you find any bugs. Thanks.