Closed tanishhshahh closed 1 year ago
After running the command: "python3 convert-pth-to-ggml.py /Users/tanish.shah/llama.cpp/models/7B/ 1" Error with sentencepiece:
Traceback (most recent call last): File "/Users/tanish.shah/llama.cpp/convert-pth-to-ggml.py", line 75, in <module> tokenizer = sentencepiece.SentencePieceProcessor(fname_tokenizer) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/tanish.shah/llama.cpp/env/lib/python3.11/site-packages/sentencepiece/__init__.py", line 447, in Init self.Load(model_file=model_file, model_proto=model_proto) File "/Users/tanish.shah/llama.cpp/env/lib/python3.11/site-packages/sentencepiece/__init__.py", line 905, in Load return self.LoadFromFile(model_file) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/tanish.shah/llama.cpp/env/lib/python3.11/site-packages/sentencepiece/__init__.py", line 310, in LoadFromFile return _sentencepiece.SentencePieceProcessor_LoadFromFile(self, arg) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ RuntimeError: Internal: /private/var/folders/rz/kp316btj1lgdlrcgdq7spdg80000gn/T/pip-install-rga_wjtr/sentencepiece_ef6dee7cfe954a50b06d772071b44d95/sentencepiece/src/sentencepiece_processor.cc(1102) [model_proto->ParseFromArray(serialized.data(), serialized.size())]
Please check whether you downloaded the model weights correctly: https://github.com/ggerganov/llama.cpp/issues/238
After running the command: "python3 convert-pth-to-ggml.py /Users/tanish.shah/llama.cpp/models/7B/ 1" Error with sentencepiece: