search

princeton-nlp / SimCSE

[EMNLP 2021] SimCSE: Simple Contrastive Learning of Sentence Embeddings https://arxiv.org/abs/2104.08821
MIT License
3.33k stars 505 forks source link

loss with hard negatives #183

Closed Golovneva closed 2 years ago

Golovneva commented 2 years ago

Can you please explain how hard negatives are incorporated in the loss function? I'm confused with the current implementation. In the code here weights is just a tensor of zeros, because z3_weight is 0.0. So when you add it to cosine similarity here it doesn't change anything, is it? So the final loss function is somewhat different from the one in the paper under eq-n (5).

Golovneva commented 2 years ago

I think I got it: it was there for ablation study on how different weights affect performance. Closing