Open lytum opened 4 years ago
hello,
in BERT tokenization.py, why are accents striped away? However, in the vocab file of multi_cased_model that supports multilingual languages, there are many accented characters.
Thanks,
+1
hello,
in BERT tokenization.py, why are accents striped away? However, in the vocab file of multi_cased_model that supports multilingual languages, there are many accented characters.
Thanks,