search

princeton-nlp / SimCSE

[EMNLP 2021] SimCSE: Simple Contrastive Learning of Sentence Embeddings https://arxiv.org/abs/2104.08821
MIT License
3.33k stars 505 forks source link

The results of the evaluation phase are not exactly in line with yours #208

Closed Moximixi closed 1 year ago

Moximixi commented 1 year ago

The command is exactly as the following example, only the model_name_or_path is changed

python evaluation.py \
    --model_name_or_path princeton-nlp/sup-simcse-bert-base-uncased \
    --pooler cls \
    --task_set sts \
    --mode test

The rightmost column is the result of my Evaluation, why some results are the same and some are different? image

github-actions[bot] commented 1 year ago

Stale issue message

gaotianyu1350 commented 1 year ago

Hi,

For the unsupervised model, you should use --pooler cls_before_pooler. Please read our README for what pooler to use for each model.