search

princeton-nlp / SimCSE

[EMNLP 2021] SimCSE: Simple Contrastive Learning of Sentence Embeddings https://arxiv.org/abs/2104.08821
MIT License
3.33k stars 505 forks source link

Uniformity and Alignment Code & Normalizing #225

Closed bruel-gabrielsson closed 1 year ago

bruel-gabrielsson commented 1 year ago

Hey! Would you be able to provide some code for the Uniformity and Alignment metrics? Because I don't see you normalizing any embeddings during training -- do you do that after training only as you compute the metric?

Thanks!

gaotianyu1350 commented 1 year ago

Hi,

We followed the original paper's code and did normalize for that evaluation. In our training we use cosine similarity, which also normalizes the embeddings.