Closed RyanHuangNLP closed 3 years ago
Hi Ryan
Can you be more specific in what is challenge you are facing in training bigbird with other languages? For example if you want to train a masked LM, we have provided the pretraining code in bigbird/pretrain/
. You should be able to use it to train for any language. You might want to use your vocabulary specific to the language.
Thanks!
@manzilz I want to train a bigbird with other language