search

princeton-nlp / SimCSE

[EMNLP 2021] SimCSE: Simple Contrastive Learning of Sentence Embeddings https://arxiv.org/abs/2104.08821
MIT License
3.37k stars 511 forks source link

unsup-SimCSE-BERTbase复现不出paper的结果 #258

Closed oufangwei closed 10 months ago

oufangwei commented 11 months ago

你好，我在复现unsup-SimCSE-BERTbase时，STS Avg.只有74.84，明显低于paper的76.25。

我使用的是： ①github上的最新的代码； ②超参数是项目下的 run_unsup_example.sh 里的设置； ③torch版本是'1.7.1+cu110'； ④GPU是单卡Nvidia 3090，CUDA Version: 11.5（12.0的也用过，结果也是74.84）； ⑤测试时，经simcse_to_huggingface.py转换，并且--pooler为cls_before_pooler

我该如何复现paper的结果，希望得到你的指点，非常感谢！

gaotianyu1350 commented 11 months ago

Hi, it might be because of the transformer version. It seems that in this issue https://github.com/princeton-nlp/SimCSE/issues/100#issuecomment-949981851, someone encountered a similar problem and was able to fix it via different transformers.

github-actions[bot] commented 10 months ago

Stale issue message