A question about the premise assumptions of the algorithm

cisnlp / simalign

Obtain Word Alignments using Pretrained Language Models (e.g., mBERT)

MIT License

345 stars 47 forks source link

A question about the premise assumptions of the algorithm #38

Closed BrightXiaoHan closed 1 year ago

BrightXiaoHan commented 1 year ago

Hi! Thanks for your work. A small question here. For a multilingual language model, why can it be guaranteed that words with the same meaning in different languages have similar representations in the word vector space? What theory is this hypothesis based on? Is there a relevant article for theoretical analysis?

pdufter commented 1 year ago

There is no guarantee for that. Also there is no strict theory behind that, but it's rather an empirical observation. See the paper for more details: https://aclanthology.org/2020.findings-emnlp.147/