search

vturrisi / solo-learn

solo-learn: a library of self-supervised methods for visual representation learning powered by Pytorch Lightning
MIT License
1.41k stars 182 forks source link

Question about the nnclr.py #252

Closed HuangChiEn closed 2 years ago

HuangChiEn commented 2 years ago

Thanks for making this SSL benchmark public, it's amazing!!

I just want to know about the detail of the nnclr.py implementation. Why do we need to define the queue as the static class variable in pl.LightningModule ?

DonkeyShot21 commented 2 years ago

Hi! Thank you for using the library.

In our implementation it is not a static class variable, it is a buffer registered in the method class. AFAIK this is the best method to ensure: 1) gradients are not propagated to the queue and it is not added to the optimizer even by mistake 2) the queue is saved in the checkpoint 3) the queue is on the right device

HuangChiEn commented 2 years ago

Hi! Thank you for using the library.

In our implementation it is not a static class variable, it is a buffer registered in the method class. AFAIK this is the best method to ensure:

  1. gradients are not propagated to the queue and it is not added to the optimizer even by mistake
  2. the queue is saved in the checkpoint
  3. the queue is on the right device

Thanks for your reply~