search

AndriyMulyar / semantic-text-similarity

an easy-to-use interface to fine-tuned BERT models for computing semantic similarity in clinical and web text. that's it.
MIT License
213 stars 51 forks source link

problem when swapped sentences #8

Open ZeinabAghahadi opened 4 years ago

ZeinabAghahadi commented 4 years ago

I load the model and try to predict the similarity between sentence A and sentence B. when I change the order of these sentences (i change the place of sentence A and B and swapped them), i get different prediction value for similarity. Why this happen?is not this measure symmetric?

AndriyMulyar commented 4 years ago

This happens because switching the sentence order will influence which pre-trained attention weights each token enters. In theory, (semantic) similarity should be symmetric - the way BERT works makes this nearly impossible.

I tried to augment the number of training instances by exploiting this symmetry. In practice, it did not lead to any real improvement in generalization.

ZeinabAghahadi commented 4 years ago

OK. I understood what you mean. Thank you so much

yuchenlin commented 4 years ago

maybe SentenceBERT can help in this case?