search

princeton-nlp / SimCSE

[EMNLP 2021] SimCSE: Simple Contrastive Learning of Sentence Embeddings https://arxiv.org/abs/2104.08821
MIT License
3.33k stars 505 forks source link

model_args.hard_negative_weight issue #224

Closed wuxiangli91 closed 1 year ago

wuxiangli91 commented 1 year ago

hi ,I find a problem that your code does not match your paper, z3_weight = cls.model_args.hard_negative_weight weights = torch.tensor( [[0.0] (cos_sim.size(-1) - z1_z3_cos.size(-1)) + [0.0] i + [z3_weight] + [0.0] * (z1_z3_cos.size(-1) - i - 1) for i in range(z1_z3_cos.size(-1))] ).to(cls.device) cos_sim = cos_sim + weights

I think hte weights should multiply cos_sim and [0.0] should replace with [1.1] ?

gaotianyu1350 commented 1 year ago

HI, here weights are applied in a log form. Weight=0 means the final weight would be e^0=1.