search

vturrisi / solo-learn

solo-learn: a library of self-supervised methods for visual representation learning powered by Pytorch Lightning
MIT License
1.38k stars 182 forks source link

Classification Loss #353

Closed mammadov7 closed 1 year ago

mammadov7 commented 1 year ago

Dear Authors,

Hope you are doing well.

In all of the pretraining methods. For example, there is a class_loss in SimCLR that doesn't exist in the original implementation. Could you please tell the reason why you have added class loss?

Best regards,

vturrisi commented 1 year ago

Hey, this has been asked many times. I think at this point it's even worth to add it to the readme.

That loss is used only to train a linear classifier together with the backbone model. We detach the features before giving them to the classifier, so the classification loss only acts on the linear classifier. This can be seen as an "online linear classifier". In our experiments, we saw that this performs very similarly to the offline linear classifier.