能否在https://huggingface.co/ 上增加一个spiece.model 文件，这样可以直接通过T5Tokenizer.from_pretrained("imxly/t5-pegasus") 进行加载？

renmada / t5-pegasus-pytorch

402 stars 61 forks source link

Closed huiyangzhou closed 3 years ago

huiyangzhou commented 3 years ago

tokenizer = T5Tokenizer.from_pretrained("imxly/t5-pegasus")#google/mt5-base

OSError: Can't load tokenizer for 'imxly/t5-pegasus'. Make sure that:

'imxly/t5-pegasus' is a correct model identifier listed on 'https://huggingface.co/models'
or 'imxly/t5-pegasus' is the correct path to a directory containing relevant tokenizer files

huiyangzhou commented 3 years ago

tokenizer = BertTokenizer.from_pretrained("imxly/t5-pegasus")

renmada commented 3 years ago

tokenizer = BertTokenizer.from_pretrained("imxly/t5-pegasus") 不能直接用BertTokenizer哦