What is 'Spearman’s rank correlation between the cosine-similarity of the sentence embeddings and the gold labels.' ?

UKPLab / sentence-transformers

State-of-the-Art Text Embeddings

https://www.sbert.net

Apache License 2.0

15.17k stars 2.47k forks source link

What is 'Spearman’s rank correlation between the cosine-similarity of the sentence embeddings and the gold labels.' ? #693

Closed Gpwner closed 3 years ago

Gpwner commented 3 years ago

In your paper,you mention this we compute the Spearman’s rank correlation between the cosine-similarity of the sentence embeddings and the gold labels. in section 4.1

Here is my question,what is the gold labels mean ,and can you provide a example to explain how to calculate the Spearman’s rank correlation in your paper?Any help will be appreciate!

nreimers commented 3 years ago

See https://github.com/UKPLab/sentence-transformers/blob/master/examples/evaluation/evaluation_stsbenchmark.py https://github.com/UKPLab/sentence-transformers/blob/master/examples/training/sts/training_stsbenchmark.py

Gold labels are provided by the dataset and were created by humans looking at the pairs and assigning a score between 0 and 5 on the similarity

Gpwner commented 3 years ago

I got it.Thanks.