facebookresearch / MUSE

A library for Multilingual Unsupervised or Supervised word Embeddings
Other
3.17k stars 544 forks source link

subword embeddings unfit with the evaluation dictionary #148

Open alwayshjia opened 4 years ago

alwayshjia commented 4 years ago

Hi, I have a question when I try to do unsupervised mapping between the source and target embeddings which are all subwords(have BPE procedures). Since the SRC and TGT embeddings are subwords and all the dictionaries are words, how can I get the evaluation dictionary that are subwords? And the question also exists when I try to do unsupervised mapping between subword embeddings.