search

DAMO-NLP-SG / VideoLLaMA2

VideoLLaMA 2: Advancing Spatial-Temporal Modeling and Audio Understanding in Video-LLMs
Apache License 2.0
871 stars 60 forks source link

Error while loading Mixtral based SFT MoE model VideoLLaMA2-8x7B: SafetensorError: Error while deserializing header: InvalidHeaderDeserialization #77

Open ApoorvFrontera opened 2 months ago

ApoorvFrontera commented 2 months ago

Hi Team,

When I am loading the Mixtral-based SFT MoE model 'DAMO-NLP-SG/VideoLLaMA2-8x7B' using the same inference code provided in the README.md, the following error is raised:

Traceback (most recent call last):
  File "/home/admin/apoorv/development/VideoLLaMA2/playground.py", line 33, in <module>
    inference()
  File "/home/admin/apoorv/development/VideoLLaMA2/playground.py", line 27, in inference
    model, processor, tokenizer = model_init(model_path)
  File "/home/admin/apoorv/development/VideoLLaMA2/videollama2/__init__.py", line 17, in model_init
    tokenizer, model, processor, context_len = load_pretrained_model(model_path, None, model_name, **kwargs)
  File "/home/admin/apoorv/development/VideoLLaMA2/videollama2/model/__init__.py", line 180, in load_pretrained_model
    model = Videollama2MixtralForCausalLM.from_pretrained(model_path, low_cpu_mem_usage=True, config=config, **kwargs)
  File "/home/admin/.conda/envs/vl2/lib/python3.10/site-packages/transformers/modeling_utils.py", line 3838, in from_pretrained
    ) = cls._load_pretrained_model(
  File "/home/admin/.conda/envs/vl2/lib/python3.10/site-packages/transformers/modeling_utils.py", line 4278, in _load_pretrained_model
    state_dict = load_state_dict(shard_file, is_quantized=is_quantized)
  File "/home/admin/.conda/envs/vl2/lib/python3.10/site-packages/transformers/modeling_utils.py", line 516, in load_state_dict
    with safe_open(checkpoint_file, framework="pt") as f:
safetensors_rust.SafetensorError: Error while deserializing header: InvalidHeaderDeserialization

I tried to find the reason for this and came across the following issues where the main reason was that the file/weights saved were not done correctly and had empty dictionaries which the safetensors can't handle. https://github.com/huggingface/transformers/issues/27397#issuecomment-1806063673

To solve this, there are some changes to be implemented before saving the model checkpoint by you guys:

  1. https://github.com/huggingface/transformers/issues/27397
  2. https://discuss.huggingface.co/t/safetensors-rust-safetenserror-while-deserializing-header-invalidheaderdeserialization/68831
  3. https://github.com/slai-labs/get-beam/issues/94
  4. https://huggingface.co/TheBloke/deepseek-coder-33B-instruct-AWQ/discussions/1
ApoorvFrontera commented 1 month ago

Hi Team Any response or help for this will be very useful.

Thanks in advance.

HugoDelhaye commented 1 month ago

I am having the same issus

williamium3000 commented 3 weeks ago

same issues here.

leoisufa commented 2 weeks ago

same issues

LiangMeng89 commented 2 days ago

same issues

Hello,I'm a phD student from ZJU, I also use videollama2 to do my own research,we create a WeChat group to discuss some issues of videollama2 and help each other,could you join us? Please contact me: WeChat number == LiangMeng19357260600, phone number == +86 19357260600,e-mail == liangmeng89@zju.edu.cn.