search

SCHENLIU / longformer-chinese

chinese version of longformer
110 stars 15 forks source link

Tokenizer #11

Closed illiterate closed 3 years ago

illiterate commented 3 years ago

您好，请问您使用的Tokenizer是哪个？LongformerTokenizer, RobertaTokenizer都不可以用，只有BertTokenizer可以用