search

antoine77340 / MIL-NCE_HowTo100M

PyTorch GPU distributed training code for MIL-NCE HowTo100M
Apache License 2.0
214 stars 31 forks source link

normalized vector dot product #10

Closed thematrixduo closed 3 years ago

thematrixduo commented 3 years ago

Hi! I noticed that in this paper you directly multiply the embedding vectors without normalizing them, as many of the recent self-supervised learning paper has done. Is there a specific reason for not doing the normalization? Thanks!

antoine77340 commented 3 years ago

Hi,

We have not seen any improvements when performing cosine similarity with temperature instead of a simple dot product.

thematrixduo commented 3 years ago

Thank you!