ValueError: `checkpoint` should be the path to a file containing a whole state dict

[fishfree]

System Info / 系統信息

NVIDIA-SMI 525.147.05 Driver Version: 525.147.05 CUDA Version: 12.0

/mnt/data/whang/miniconda3/envs/cogvlm2/bin/python

Python 3.11.9

git pull the latest repo.

Who can help? / 谁可以帮助到您？

python cli_demo_multi_gpus.py 时报错如下：

省略了很多类似的warning等信息。。。
/mnt/data/whang/miniconda3/envs/cogvlm2/lib/python3.11/site-packages/torch/nn/modules/module.py:2047: UserWarning: for model.vision.transformer.layers.60.post_attention_layernorm.weight: copying from a non-meta parameter in the checkpoint to a meta parameter in the current model, which is a no-op. (Did you mean to pass `assign=True` to assign items in the state dictionary to their corresponding key in the module instead of copying them in place?)
  warnings.warn(f'for {key}: copying from a non-meta parameter in the checkpoint to a meta '
/mnt/data/whang/miniconda3/envs/cogvlm2/lib/python3.11/site-packages/torch/nn/modules/module.py:2047: UserWarning: for model.vision.transformer.layers.60.post_attention_layernorm.bias: copying from a non-meta parameter in the checkpoint to a meta parameter in the current model, which is a no-op. (Did you mean to pass `assign=True` to assign items in the state dictionary to their corresponding key in the module instead of copying them in place?)
  warnings.warn(f'for {key}: copying from a non-meta parameter in the checkpoint to a meta '
/mnt/data/whang/miniconda3/envs/cogvlm2/lib/python3.11/site-packages/torch/nn/modules/module.py:2047: UserWarning: for model.vision.transformer.layers.61.input_layernorm.weight: copying from a non-meta parameter in the checkpoint to a meta parameter in the current model, which is a no-op. (Did you mean to pass `assign=True` to assign items in the state dictionary to their corresponding key in the module instead of copying them in place?)
  warnings.warn(f'for {key}: copying from a non-meta parameter in the checkpoint to a meta '
/mnt/data/whang/miniconda3/envs/cogvlm2/lib/python3.11/site-packages/torch/nn/modules/module.py:2047: UserWarning: for model.vision.transformer.layers.61.input_layernorm.bias: copying from a non-meta parameter in the checkpoint to a meta parameter in the current model, which is a no-op. (Did you mean to pass `assign=True` to assign items in the state dictionary to their corresponding key in the module instead of copying them in place?)
  warnings.warn(f'for {key}: copying from a non-meta parameter in the checkpoint to a meta '
/mnt/data/whang/miniconda3/envs/cogvlm2/lib/python3.11/site-packages/torch/nn/modules/module.py:2047: UserWarning: for model.vision.transformer.layers.61.attention.query_key_value.weight: copying from a non-meta parameter in the checkpoint to a meta parameter in the current model, which is a no-op. (Did you mean to pass `assign=True` to assign items in the state dictionary to their corresponding key in the module instead of copying them in place?)
  warnings.warn(f'for {key}: copying from a non-meta parameter in the checkpoint to a meta '
/mnt/data/whang/miniconda3/envs/cogvlm2/lib/python3.11/site-packages/torch/nn/modules/module.py:2047: UserWarning: for model.vision.transformer.layers.61.attention.query_key_value.bias: copying from a non-meta parameter in the checkpoint to a meta parameter in the current model, which is a no-op. (Did you mean to pass `assign=True` to assign items in the state dictionary to their corresponding key in the module instead of copying them in place?)
  warnings.warn(f'for {key}: copying from a non-meta parameter in the checkpoint to a meta '
/mnt/data/whang/miniconda3/envs/cogvlm2/lib/python3.11/site-packages/torch/nn/modules/module.py:2047: UserWarning: for model.vision.transformer.layers.61.attention.dense.weight: copying from a non-meta parameter in the checkpoint to a meta parameter in the current model, which is a no-op. (Did you mean to pass `assign=True` to assign items in the state dictionary to their corresponding key in the module instead of copying them in place?)
  warnings.warn(f'for {key}: copying from a non-meta parameter in the checkpoint to a meta '
/mnt/data/whang/miniconda3/envs/cogvlm2/lib/python3.11/site-packages/torch/nn/modules/module.py:2047: UserWarning: for model.vision.transformer.layers.61.attention.dense.bias: copying from a non-meta parameter in the checkpoint to a meta parameter in the current model, which is a no-op. (Did you mean to pass `assign=True` to assign items in the state dictionary to their corresponding key in the module instead of copying them in place?)
  warnings.warn(f'for {key}: copying from a non-meta parameter in the checkpoint to a meta '
/mnt/data/whang/miniconda3/envs/cogvlm2/lib/python3.11/site-packages/torch/nn/modules/module.py:2047: UserWarning: for model.vision.transformer.layers.61.mlp.fc1.weight: copying from a non-meta parameter in the checkpoint to a meta parameter in the current model, which is a no-op. (Did you mean to pass `assign=True` to assign items in the state dictionary to their corresponding key in the module instead of copying them in place?)
  warnings.warn(f'for {key}: copying from a non-meta parameter in the checkpoint to a meta '
/mnt/data/whang/miniconda3/envs/cogvlm2/lib/python3.11/site-packages/torch/nn/modules/module.py:2047: UserWarning: for model.vision.transformer.layers.61.mlp.fc1.bias: copying from a non-meta parameter in the checkpoint to a meta parameter in the current model, which is a no-op. (Did you mean to pass `assign=True` to assign items in the state dictionary to their corresponding key in the module instead of copying them in place?)
  warnings.warn(f'for {key}: copying from a non-meta parameter in the checkpoint to a meta '
/mnt/data/whang/miniconda3/envs/cogvlm2/lib/python3.11/site-packages/torch/nn/modules/module.py:2047: UserWarning: for model.vision.transformer.layers.61.mlp.fc2.weight: copying from a non-meta parameter in the checkpoint to a meta parameter in the current model, which is a no-op. (Did you mean to pass `assign=True` to assign items in the state dictionary to their corresponding key in the module instead of copying them in place?)
  warnings.warn(f'for {key}: copying from a non-meta parameter in the checkpoint to a meta '
/mnt/data/whang/miniconda3/envs/cogvlm2/lib/python3.11/site-packages/torch/nn/modules/module.py:2047: UserWarning: for model.vision.transformer.layers.61.mlp.fc2.bias: copying from a non-meta parameter in the checkpoint to a meta parameter in the current model, which is a no-op. (Did you mean to pass `assign=True` to assign items in the state dictionary to their corresponding key in the module instead of copying them in place?)
  warnings.warn(f'for {key}: copying from a non-meta parameter in the checkpoint to a meta '
/mnt/data/whang/miniconda3/envs/cogvlm2/lib/python3.11/site-packages/torch/nn/modules/module.py:2047: UserWarning: for model.vision.transformer.layers.61.post_attention_layernorm.weight: copying from a non-meta parameter in the checkpoint to a meta parameter in the current model, which is a no-op. (Did you mean to pass `assign=True` to assign items in the state dictionary to their corresponding key in the module instead of copying them in place?)
  warnings.warn(f'for {key}: copying from a non-meta parameter in the checkpoint to a meta '
/mnt/data/whang/miniconda3/envs/cogvlm2/lib/python3.11/site-packages/torch/nn/modules/module.py:2047: UserWarning: for model.vision.transformer.layers.61.post_attention_layernorm.bias: copying from a non-meta parameter in the checkpoint to a meta parameter in the current model, which is a no-op. (Did you mean to pass `assign=True` to assign items in the state dictionary to their corresponding key in the module instead of copying them in place?)
  warnings.warn(f'for {key}: copying from a non-meta parameter in the checkpoint to a meta '
/mnt/data/whang/miniconda3/envs/cogvlm2/lib/python3.11/site-packages/torch/nn/modules/module.py:2047: UserWarning: for model.vision.transformer.layers.62.input_layernorm.weight: copying from a non-meta parameter in the checkpoint to a meta parameter in the current model, which is a no-op. (Did you mean to pass `assign=True` to assign items in the state dictionary to their corresponding key in the module instead of copying them in place?)
  warnings.warn(f'for {key}: copying from a non-meta parameter in the checkpoint to a meta '
/mnt/data/whang/miniconda3/envs/cogvlm2/lib/python3.11/site-packages/torch/nn/modules/module.py:2047: UserWarning: for model.vision.transformer.layers.62.input_layernorm.bias: copying from a non-meta parameter in the checkpoint to a meta parameter in the current model, which is a no-op. (Did you mean to pass `assign=True` to assign items in the state dictionary to their corresponding key in the module instead of copying them in place?)
  warnings.warn(f'for {key}: copying from a non-meta parameter in the checkpoint to a meta '
/mnt/data/whang/miniconda3/envs/cogvlm2/lib/python3.11/site-packages/torch/nn/modules/module.py:2047: UserWarning: for model.vision.transformer.layers.62.attention.query_key_value.weight: copying from a non-meta parameter in the checkpoint to a meta parameter in the current model, which is a no-op. (Did you mean to pass `assign=True` to assign items in the state dictionary to their corresponding key in the module instead of copying them in place?)
  warnings.warn(f'for {key}: copying from a non-meta parameter in the checkpoint to a meta '
/mnt/data/whang/miniconda3/envs/cogvlm2/lib/python3.11/site-packages/torch/nn/modules/module.py:2047: UserWarning: for model.vision.transformer.layers.62.attention.query_key_value.bias: copying from a non-meta parameter in the checkpoint to a meta parameter in the current model, which is a no-op. (Did you mean to pass `assign=True` to assign items in the state dictionary to their corresponding key in the module instead of copying them in place?)
  warnings.warn(f'for {key}: copying from a non-meta parameter in the checkpoint to a meta '
/mnt/data/whang/miniconda3/envs/cogvlm2/lib/python3.11/site-packages/torch/nn/modules/module.py:2047: UserWarning: for model.vision.transformer.layers.62.attention.dense.weight: copying from a non-meta parameter in the checkpoint to a meta parameter in the current model, which is a no-op. (Did you mean to pass `assign=True` to assign items in the state dictionary to their corresponding key in the module instead of copying them in place?)
  warnings.warn(f'for {key}: copying from a non-meta parameter in the checkpoint to a meta '
/mnt/data/whang/miniconda3/envs/cogvlm2/lib/python3.11/site-packages/torch/nn/modules/module.py:2047: UserWarning: for model.vision.transformer.layers.62.attention.dense.bias: copying from a non-meta parameter in the checkpoint to a meta parameter in the current model, which is a no-op. (Did you mean to pass `assign=True` to assign items in the state dictionary to their corresponding key in the module instead of copying them in place?)
  warnings.warn(f'for {key}: copying from a non-meta parameter in the checkpoint to a meta '
/mnt/data/whang/miniconda3/envs/cogvlm2/lib/python3.11/site-packages/torch/nn/modules/module.py:2047: UserWarning: for model.vision.transformer.layers.62.mlp.fc1.weight: copying from a non-meta parameter in the checkpoint to a meta parameter in the current model, which is a no-op. (Did you mean to pass `assign=True` to assign items in the state dictionary to their corresponding key in the module instead of copying them in place?)
  warnings.warn(f'for {key}: copying from a non-meta parameter in the checkpoint to a meta '
/mnt/data/whang/miniconda3/envs/cogvlm2/lib/python3.11/site-packages/torch/nn/modules/module.py:2047: UserWarning: for model.vision.transformer.layers.62.mlp.fc1.bias: copying from a non-meta parameter in the checkpoint to a meta parameter in the current model, which is a no-op. (Did you mean to pass `assign=True` to assign items in the state dictionary to their corresponding key in the module instead of copying them in place?)
  warnings.warn(f'for {key}: copying from a non-meta parameter in the checkpoint to a meta '
/mnt/data/whang/miniconda3/envs/cogvlm2/lib/python3.11/site-packages/torch/nn/modules/module.py:2047: UserWarning: for model.vision.transformer.layers.62.mlp.fc2.weight: copying from a non-meta parameter in the checkpoint to a meta parameter in the current model, which is a no-op. (Did you mean to pass `assign=True` to assign items in the state dictionary to their corresponding key in the module instead of copying them in place?)
  warnings.warn(f'for {key}: copying from a non-meta parameter in the checkpoint to a meta '
/mnt/data/whang/miniconda3/envs/cogvlm2/lib/python3.11/site-packages/torch/nn/modules/module.py:2047: UserWarning: for model.vision.transformer.layers.62.mlp.fc2.bias: copying from a non-meta parameter in the checkpoint to a meta parameter in the current model, which is a no-op. (Did you mean to pass `assign=True` to assign items in the state dictionary to their corresponding key in the module instead of copying them in place?)
  warnings.warn(f'for {key}: copying from a non-meta parameter in the checkpoint to a meta '
/mnt/data/whang/miniconda3/envs/cogvlm2/lib/python3.11/site-packages/torch/nn/modules/module.py:2047: UserWarning: for model.vision.transformer.layers.62.post_attention_layernorm.weight: copying from a non-meta parameter in the checkpoint to a meta parameter in the current model, which is a no-op. (Did you mean to pass `assign=True` to assign items in the state dictionary to their corresponding key in the module instead of copying them in place?)
  warnings.warn(f'for {key}: copying from a non-meta parameter in the checkpoint to a meta '
/mnt/data/whang/miniconda3/envs/cogvlm2/lib/python3.11/site-packages/torch/nn/modules/module.py:2047: UserWarning: for model.vision.transformer.layers.62.post_attention_layernorm.bias: copying from a non-meta parameter in the checkpoint to a meta parameter in the current model, which is a no-op. (Did you mean to pass `assign=True` to assign items in the state dictionary to their corresponding key in the module instead of copying them in place?)
  warnings.warn(f'for {key}: copying from a non-meta parameter in the checkpoint to a meta '
/mnt/data/whang/miniconda3/envs/cogvlm2/lib/python3.11/site-packages/torch/nn/modules/module.py:2047: UserWarning: for model.vision.linear_proj.linear_proj.weight: copying from a non-meta parameter in the checkpoint to a meta parameter in the current model, which is a no-op. (Did you mean to pass `assign=True` to assign items in the state dictionary to their corresponding key in the module instead of copying them in place?)
  warnings.warn(f'for {key}: copying from a non-meta parameter in the checkpoint to a meta '
/mnt/data/whang/miniconda3/envs/cogvlm2/lib/python3.11/site-packages/torch/nn/modules/module.py:2047: UserWarning: for model.vision.linear_proj.norm1.weight: copying from a non-meta parameter in the checkpoint to a meta parameter in the current model, which is a no-op. (Did you mean to pass `assign=True` to assign items in the state dictionary to their corresponding key in the module instead of copying them in place?)
  warnings.warn(f'for {key}: copying from a non-meta parameter in the checkpoint to a meta '
/mnt/data/whang/miniconda3/envs/cogvlm2/lib/python3.11/site-packages/torch/nn/modules/module.py:2047: UserWarning: for model.vision.linear_proj.norm1.bias: copying from a non-meta parameter in the checkpoint to a meta parameter in the current model, which is a no-op. (Did you mean to pass `assign=True` to assign items in the state dictionary to their corresponding key in the module instead of copying them in place?)
  warnings.warn(f'for {key}: copying from a non-meta parameter in the checkpoint to a meta '
/mnt/data/whang/miniconda3/envs/cogvlm2/lib/python3.11/site-packages/torch/nn/modules/module.py:2047: UserWarning: for model.vision.linear_proj.dense_h_to_4h.weight: copying from a non-meta parameter in the checkpoint to a meta parameter in the current model, which is a no-op. (Did you mean to pass `assign=True` to assign items in the state dictionary to their corresponding key in the module instead of copying them in place?)
  warnings.warn(f'for {key}: copying from a non-meta parameter in the checkpoint to a meta '
/mnt/data/whang/miniconda3/envs/cogvlm2/lib/python3.11/site-packages/torch/nn/modules/module.py:2047: UserWarning: for model.vision.linear_proj.gate_proj.weight: copying from a non-meta parameter in the checkpoint to a meta parameter in the current model, which is a no-op. (Did you mean to pass `assign=True` to assign items in the state dictionary to their corresponding key in the module instead of copying them in place?)
  warnings.warn(f'for {key}: copying from a non-meta parameter in the checkpoint to a meta '
/mnt/data/whang/miniconda3/envs/cogvlm2/lib/python3.11/site-packages/torch/nn/modules/module.py:2047: UserWarning: for model.vision.linear_proj.dense_4h_to_h.weight: copying from a non-meta parameter in the checkpoint to a meta parameter in the current model, which is a no-op. (Did you mean to pass `assign=True` to assign items in the state dictionary to their corresponding key in the module instead of copying them in place?)
  warnings.warn(f'for {key}: copying from a non-meta parameter in the checkpoint to a meta '
/mnt/data/whang/miniconda3/envs/cogvlm2/lib/python3.11/site-packages/torch/nn/modules/module.py:2047: UserWarning: for model.vision.conv.weight: copying from a non-meta parameter in the checkpoint to a meta parameter in the current model, which is a no-op. (Did you mean to pass `assign=True` to assign items in the state dictionary to their corresponding key in the module instead of copying them in place?)
  warnings.warn(f'for {key}: copying from a non-meta parameter in the checkpoint to a meta '
/mnt/data/whang/miniconda3/envs/cogvlm2/lib/python3.11/site-packages/torch/nn/modules/module.py:2047: UserWarning: for model.vision.conv.bias: copying from a non-meta parameter in the checkpoint to a meta parameter in the current model, which is a no-op. (Did you mean to pass `assign=True` to assign items in the state dictionary to their corresponding key in the module instead of copying them in place?)
  warnings.warn(f'for {key}: copying from a non-meta parameter in the checkpoint to a meta '
/mnt/data/whang/miniconda3/envs/cogvlm2/lib/python3.11/site-packages/torch/nn/modules/module.py:2047: UserWarning: for lm_head.weight: copying from a non-meta parameter in the checkpoint to a meta parameter in the current model, which is a no-op. (Did you mean to pass `assign=True` to assign items in the state dictionary to their corresponding key in the module instead of copying them in place?)
  warnings.warn(f'for {key}: copying from a non-meta parameter in the checkpoint to a meta '
Loading checkpoint shards: 100%|█████████████████████| 8/8 [00:05<00:00,  1.37it/s]
Traceback (most recent call last):
  File "/mnt/data/whang/CogVLM2/basic_demo/cli_demo_multi_gpus.py", line 51, in <module>
    model = load_checkpoint_and_dispatch(model, MODEL_PATH, device_map=device_map, dtype=TORCH_TYPE)
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/mnt/data/whang/miniconda3/envs/cogvlm2/lib/python3.11/site-packages/accelerate/big_modeling.py", line 607, in load_checkpoint_and_dispatch
    load_checkpoint_in_model(
  File "/mnt/data/whang/miniconda3/envs/cogvlm2/lib/python3.11/site-packages/accelerate/utils/modeling.py", line 1647, in load_checkpoint_in_model
    raise ValueError(
ValueError: `checkpoint` should be the path to a file containing a whole state dict, or the index of a sharded checkpoint, or a folder containing a sharded checkpoint or the whole state dict, but got THUDM/cogvlm2-llama3-chat-19B.

Information / 问题信息

• [X] The official example scripts / 官方的示例脚本
• [ ] My own modified scripts / 我自己修改的脚本和任务

Reproduction / 复现过程

git pull https://github.com/THUDM/CogVLM2 cd CogVLM2 conda create -n cogvlm2 python=3.11 -y conda activate cogvlm2 cd basic_demo python -m pip install -r requirements python -m pip install accelerate python cli_demo_multi_gpus.py

Expected behavior / 期待表现

当然是不报错、正常运行了。

[zRzRzRzRzRzRzR]

THUDM/cogvlm2-llama3-chat-19B 换成你本地的绝对路径，记得看微调的readme，一堆参数要自己设置

[elloza]

THUDM/cogvlm2-llama3-chat-19B 换成你本地的绝对路径，记得看微调的readme，一堆参数要自己设置

So... what is the right local path exactly?

[philmod-h]

I'm getting the same error too when running

$ python cli_demo_multi_gpus.py

Loading checkpoint shards: 100%|████████████████████████████████████████████████████████████████████████████| 8/8 [00:00<00:00, 10.22it/s]
Traceback (most recent call last):
  File "<REDACTED>/code/CogVLM2/basic_demo/cli_demo_multi_gpus.py", line 57, in <module>
    model = load_checkpoint_and_dispatch(
  File "<REDACTED>/miniconda3/envs/cogvlm/lib/python3.10/site-packages/accelerate/big_modeling.py", line 613, in load_checkpoint_and_dispatch
    load_checkpoint_in_model(
  File "<REDACTED>/miniconda3/envs/cogvlm/lib/python3.10/site-packages/accelerate/utils/modeling.py", line 1732, in load_checkpoint_in_model
    raise ValueError(
ValueError: `checkpoint` should be the path to a file containing a whole state dict, or the index of a sharded checkpoint, or a folder containing a sharded checkpoint or the whole state dict, but got THUDM/cogvlm2-llama3-chat-19B.

Any help would be appreciated :)