search

princeton-nlp / SimCSE

[EMNLP 2021] SimCSE: Simple Contrastive Learning of Sentence Embeddings https://arxiv.org/abs/2104.08821
MIT License
3.35k stars 507 forks source link

A little question about evaluation #94

Closed shadowkiller33 closed 2 years ago

shadowkiller33 commented 2 years ago

Amazing work ^^ and thanks for your beautiful codes, I can easily reproduce the results. However, I got a small question, since the results in STS are reported averagly, how can I see the performance on each STS task?

Can't be more appreciated for your swift reply.

gaotianyu1350 commented 2 years ago

Hi,

If you use our evaluation code, it will output a table like this:

------ test ------
+-------+-------+-------+-------+-------+--------------+-----------------+-------+
| STS12 | STS13 | STS14 | STS15 | STS16 | STSBenchmark | SICKRelatedness |  Avg. |
+-------+-------+-------+-------+-------+--------------+-----------------+-------+
| 75.30 | 84.67 | 80.19 | 85.40 | 80.82 |    84.26     |      80.39      | 81.58 |
+-------+-------+-------+-------+-------+--------------+-----------------+-------+

which shows the detailed performance of each task.