search

SCHENLIU / longformer-chinese

chinese version of longformer
110 stars 15 forks source link

Should I use RobertaTokenizer as readme said when I load longformer-chinese model? #12

Open jxyxiangyu opened 2 years ago

jxyxiangyu commented 2 years ago

As title said, or should I use which tokenizer?