search

google-research / simclr

SimCLRv2 - Big Self-Supervised Models are Strong Semi-Supervised Learners
https://arxiv.org/abs/2006.10029
Apache License 2.0
4.07k stars 622 forks source link

Constrastive loss input format in the code #158

Open Darthholi opened 3 years ago

Darthholi commented 3 years ago

Hello, thanks for the wonderful work.

Looking at the contrastive loss (https://github.com/google-research/simclr/blob/master/tf2/objective.py, add_contrastive_loss(hidden,...) - please, what is the order of (hidden representation of) items in the batch?

Is it [positive, positive, positive, negative, negative, negative] or [positive, negative, positive, negative, positive, negative], or something different? I have read the article and code and need some reassurance. (By positive I mean for example the same augmented datapoint, by negative I mean completely different image).

Thank You!

chentingpc commented 3 years ago

hidden1 and hidden2 are both in (bsz, dim) so corresponding rows in hidden1 and hidden2 are positive.