Error in loading Video-LLaMA-2-7b_Finetuned

the error occurs in

                self.llama_model = LlamaForCausalLM.from_pretrained(
                    llama_model,
                    torch_dtype=torch.bfloat16,
                )

the the error is

INFO:root:load checkpoint from ckpt/instruct-blip/instruct_blip_vicuna7b_trimmed.pth
INFO:root:freeze Qformer
INFO:root:Loading Q-Former Done
INFO:root:Loading LLAMA Tokenizer
You are using the default legacy behaviour of the <class 'transformers.models.llama.tokenization_llama.LlamaTokenizer'>. This is expected, and simply means that the `legacy` (previous) behavior will be used so nothing changes for you. If you want to use the new behaviour, set `legacy=False`. This should only be set if you understand what it means, and thoroughly read the reason why this was added as explained in https://github.com/huggingface/transformers/pull/24565
INFO:root:Loading LLAMA Model
Loading checkpoint shards:   0%|                                                          | 0/2 [00:00<?, ?it/s]
Traceback (most recent call last):
  File "/media/sdb/long/conda/envs/timechat/lib/python3.9/site-packages/transformers/modeling_utils.py", line 533, in load_state_dict
    return torch.load(
  File "/media/sdb/long/conda/envs/timechat/lib/python3.9/site-packages/torch/serialization.py", line 712, in load
    return _load(opened_zipfile, map_location, pickle_module, **pickle_load_args)
  File "/media/sdb/long/conda/envs/timechat/lib/python3.9/site-packages/torch/serialization.py", line 1047, in _load
    unpickler = UnpicklerWrapper(data_file, **pickle_load_args)
TypeError: 'weights_only' is an invalid keyword argument for Unpickler()

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/media/sdb/long/conda/envs/timechat/lib/python3.9/site-packages/transformers/modeling_utils.py", line 542, in load_state_dict
    if f.read(7) == "version":
  File "/media/sdb/long/conda/envs/timechat/lib/python3.9/codecs.py", line 322, in decode
    (result, consumed) = self._buffer_decode(data, self.errors, final)
UnicodeDecodeError: 'utf-8' codec can't decode byte 0x80 in position 128: invalid start byte

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/media/sdb/long/lmm/TimeChat/demo.py", line 68, in <module>
    model = model_cls.from_config(model_config).to('cuda:{}'.format(args.gpu_id))
  File "/media/sdb/long/lmm/TimeChat/timechat/models/timechat.py", line 549, in from_config
    model = cls(
  File "/media/sdb/long/lmm/TimeChat/timechat/models/timechat.py", line 159, in __init__
    self.llama_model = LlamaForCausalLM.from_pretrained(
  File "/media/sdb/long/conda/envs/timechat/lib/python3.9/site-packages/transformers/modeling_utils.py", line 3852, in from_pretrained
    ) = cls._load_pretrained_model(
  File "/media/sdb/long/conda/envs/timechat/lib/python3.9/site-packages/transformers/modeling_utils.py", line 4261, in _load_pretrained_model
    state_dict = load_state_dict(shard_file)
  File "/media/sdb/long/conda/envs/timechat/lib/python3.9/site-packages/transformers/modeling_utils.py", line 554, in load_state_dict
    raise OSError(
OSError: Unable to load weights from pytorch checkpoint file for 'ckpt/Video-LLaMA-2-7B-Finetuned/llama-2-7b-chat-hf/pytorch_model-00001-of-00002.bin' at 'ckpt/Video-LLaMA-2-7B-Finetuned/llama-2-7b-chat-hf/pytorch_model-00001-of-00002.bin'. If you tried to load a PyTorch model from a TF 2.0 checkpoint, please set from_tf=True.

RenShuhuai-Andy / TimeChat

Error in loading Video-LLaMA-2-7b_Finetuned #7