search

SeanLee97 / AnglE

Train and Infer Powerful Sentence Embeddings with AnglE | 🔥 SOTA on STS and MTEB Leaderboard
https://arxiv.org/abs/2309.12871
MIT License
493 stars 33 forks source link

AngIE motivation? #70

Open xiewen354 opened 7 months ago

xiewen354 commented 7 months ago

Hi author, I have a question: the so-called cosine similarity is actually a vector dot product, and there is no real cosine at all. When the gradient is calculated, there is only multiplication, there is no cos at all, and there is no so-called saturation region where the gradient disappears. Can you explain?

Toshimonster commented 6 months ago

But it is equivalent to a cosine inside of an inner product space, of which we are.