can not convert llama3-8b-instrct to tubomind by imdeploy;RuntimeError: Internal: src/sentencepiece_processor.cc(1101) [model_proto->ParseFromArray(serialized.data(), serialized.size())] error #615

meta-llama / llama3

The official Meta Llama 3 GitHub site

Other

26.18k stars 2.94k forks source link

can not convert llama3-8b-instrct to tubomind by imdeploy;RuntimeError: Internal: src/sentencepiece_processor.cc(1101) [model_proto->ParseFromArray(serialized.data(), serialized.size())] error #615 #230

Open liutao053877 opened 3 months ago

liutao053877 commented 3 months ago

the snapshot is

I wonder is the tokenizer.model is not rigth? But I download it from https://huggingface.co/meta-llama/Meta-Llama-3-8B-Instruct/tree/main/original.

How can i fix this problem? thx

liutao053877 commented 3 months ago

the error information is : RuntimeError: Internal: src/sentencepiece_processor.cc(1101) [model_proto->ParseFromArray(serialized.data(), serialized.size())]

It means maybe the tokenizer.model is not ok

XiangwenXiao commented 3 months ago

你好，请问解决这个问题了吗，我也报了同样的错误

baiyuting commented 2 months ago