tomaarsen
commented
1 month ago Hello!
Great question! There was a similar question on SetFit (a project that finetunes SentenceTransformer models for classification) about this: https://github.com/huggingface/setfit/issues/254 There seem to be two parts to consider:
- In theory: a value of -1 means "opposite", whereas 0 means "orthogonal" or unrelated. Generally in Sentence Transformers, we want the train models to give us an idea of similarity/relatedness, and we don't care much for the opposite.
- In practice: using ranges of 0 to 1 simply seems to result in higher scores. There's also a lot more training data where the lowest score is "unrelated", rather than "opposite".
I believe you can specify labels in the range of (-1, 1) to the loss, so you're actually free to run some experiments yourself. For example, you can use this STSb training script first normally, and then with the labels remapped to (-1, 1). (Hint: label * 2 - 1 should do it)
- Tom Aarsen
Hi! I hope you are doing well!
I am a newbie and here I am doing a prediction task using cosine similarity, in CosineSimilarityLoss (https://github.com/UKPLab/sentence-transformers/blob/master/sentence_transformers/losses/CosineSimilarityLoss.py) It seems didn't normalize the cosine similarity result to (0,1) (hope I understood it correctly), so the cosine similarity result would still has the range (-1, 1) may I ask why it still accepts the similarity score from 0 to 1 as a label? @tomaarsen
Thanks! :)