bheinzerling / bpemb

Pre-trained subword embeddings in 275 languages, based on Byte-Pair Encoding (BPE)
https://nlp.h-its.org/bpemb
MIT License
1.18k stars 101 forks source link

Issues after updating to gensim 4.0.0 #56

Closed arun5309 closed 3 years ago

arun5309 commented 3 years ago

One of the simple example that illustrates this issue is the pieces property of an bpemb.BPEmb instance throws the error "AttributeError: The index2word attribute has been replaced by index_to_key since Gensim 4.0.0. See https://github.com/RaRe-Technologies/gensim/wiki/Migrating-from-Gensim-3.x-to-4".

So, support for gensim 4.0.0 is needed.

bheinzerling commented 3 years ago

Thanks for raising this issue. I updated the code and according to the unit tests everything should work with gensim 4.0.0 now. Let me know if you encounter any problem.