search

HobbitLong / CMC

[arXiv 2019] "Contrastive Multiview Coding", also contains implementations for MoCo and InstDis
BSD 2-Clause "Simplified" License
1.3k stars 179 forks source link

Question about softmax loss #40

Closed hyBlue closed 4 years ago

hyBlue commented 4 years ago

https://github.com/HobbitLong/CMC/blob/58d06e9a82f7fea2e4af0a251726e9c6bf67c7c9/NCE/NCECriterion.py#L35-L46

Hi, I have a question about using softmax instead of NCE loss. In that function, every label is set zero including the critic value of positive sample, which has index 0 of the batch. I want to know the reason. My take on this is that the label should be [1, 0, 0, 0, ...]. Isn't it?

hyBlue commented 4 years ago

I'm an idiot

vinsis commented 4 years ago

@hyBlue seems you found out the answer yourself. Could you explain what NCESoftmaxLoss is trying to do here? I am having the same doubt as you initially had (and feeling like an idiot).

vinsis commented 4 years ago

Explanation: label = torch.zeros([bsz]).cuda().long() means the labels are [1, 0, 0, 0, ...] for each sample. label specifies the label of the positive sample, which is 0 in each case.

It's funny how easy it is to get confused even with something you think you are fairly familiar with.

hyBlue commented 4 years ago

@vinsis Great! You're right

zhyever commented 3 years ago

Thanks a lot! (A new idiot)

jessico commented 2 years ago

triple idiot.