Closed ghost closed 2 years ago
Hi, the batch consists of two subsets - x0
and x1
. Adding logits00
after logits01
means using pairs from x0 as negatives. The same approach you can see here https://github.com/google-research/simclr/blob/master/objective.py#L83
Subtraction of the max
on the last line does not change the result of cross_entropy
, it's only for numerical stability.
Hi,
Thanks for such an interesting technique. I have a question about Contrastive Loss. Could you please explain the rationale behind the following code snippet -
I understand this resembles InfoNCE Loss but I am unable to understand why would you concat two logits and then take max of the resultant matrix.
Thanks