search

soulteary / docker-llama2-chat

Play LLaMA2 (official / 中文版 / INT4 / llama2.cpp) Together! ONLY 3 STEPS! ( non GPU / 5GB vRAM / 8~14GB vRAM)
https://www.zhihu.com/people/soulteary/posts
Apache License 2.0
528 stars 82 forks source link

HeaderTooLarge when testing #2

Closed DYSpider13 closed 1 year ago

DYSpider13 commented 1 year ago

I tried deploying it. I had this error:

Traceback (most recent call last): File "/app/model.py", line 10, in model = AutoModelForCausalLM.from_pretrained( File "/usr/local/lib/python3.10/dist-packages/transformers/models/auto/auto_factory.py", line 493, in from_pretrained return model_class.from_pretrained( File "/usr/local/lib/python3.10/dist-packages/transformers/modeling_utils.py", line 2903, in from_pretrained ) = cls._load_pretrained_model( File "/usr/local/lib/python3.10/dist-packages/transformers/modeling_utils.py", line 3246, in _load_pretrained_model state_dict = load_state_dict(shard_file) File "/usr/local/lib/python3.10/dist-packages/transformers/modeling_utils.py", line 447, in load_state_dict with safe_open(checkpoint_file, framework="pt") as f: safetensors_rust.SafetensorError: Error while deserializing header: HeaderTooLarge

Any idea ?

soulteary commented 1 year ago

same as #1

Please make sure your model file path is correct, you can refer to the following article

https://soulteary.com/2023/07/21/use-docker-to-quickly-get-started-with-the-official-version-of-llama2-open-source-large-model.html