search

X-PLUG / mPLUG-Owl

mPLUG-Owl: The Powerful Multi-modal Large Language Model Family
https://www.modelscope.cn/studios/damo/mPLUG-Owl
MIT License
2.25k stars 171 forks source link

tokenizer load失败 #91

Closed dingtine closed 1 year ago

dingtine commented 1 year ago

用MplugOwlTokenizer报： The tokenizer class you load from this checkpoint is not the same type as the class this function is called from. It may result in unexpected tokenization. The tokenizer class you load from this checkpoint is 'LlamaTokenizer'. The class this function is called from is 'MplugOwlTokenizer'.

改成LlamaTokenizer报： not a string

MAGAer13 commented 1 year ago

Try to pull the latest code and the latest checkpoint on huggingface modelhub. The MplugOwlTokenizer is just inherited from LlamaTokenizer.

neverstoplearn commented 1 year ago

Try to pull the latest code and the latest checkpoint on huggingface modelhub. The MplugOwlTokenizer is just inherited from LlamaTokenizer.

IS this repo could fintune mutlilanguage model?

I got this error:

The tokenizer class you load from this checkpoint is not the same type as the class this function is called from. It may result in unexpected tokenization. The tokenizer class you load from this checkpoint is 'BloomTokenizerFast'. The class this function is called from is 'MplugOwlTokenizer'.

the code and the model are the latest. thanks.

LinB203 commented 1 year ago

same question. is it normal?