Closed HuangChiEn closed 2 years ago
Hi! Thank you for using the library.
In our implementation it is not a static class variable, it is a buffer registered in the method class. AFAIK this is the best method to ensure: 1) gradients are not propagated to the queue and it is not added to the optimizer even by mistake 2) the queue is saved in the checkpoint 3) the queue is on the right device
Hi! Thank you for using the library.
In our implementation it is not a static class variable, it is a buffer registered in the method class. AFAIK this is the best method to ensure:
- gradients are not propagated to the queue and it is not added to the optimizer even by mistake
- the queue is saved in the checkpoint
- the queue is on the right device
Thanks for your reply~
Thanks for making this SSL benchmark public, it's amazing!!
I just want to know about the detail of the nnclr.py implementation. Why do we need to define the queue as the static class variable in pl.LightningModule ?