Need either a `state_dict` or a `save_folder` containing offloaded weights.

[zhaodaojie]

Describe the issue

warnings.warn(f'for {key}: copying from a non-meta parameter in the checkpoint to a meta ' Loading checkpoint shards: 100%|██████████| 6/6 [02:41<00:00, 26.98s/it] You are using a model of type llava to instantiate a model of type llava_llama. This is not supported for all configurations of models and can yield errors. Loading checkpoint shards: 100%|██████████| 6/6 [05:39<00:00, 56.56s/it] Traceback (most recent call last): File "/data2/zhaodaojie/project/LLaVA/ceshi/ceshi1.py", line 36, in

File "/data2/zhaodaojie/project/LLaVA/llava/eval/run_llava.py", line 55, in eval_model tokenizer, model, image_processor, context_len = load_pretrained_model( File "/data2/zhaodaojie/project/LLaVA/llava/model/builder.py", line 117, in load_pretrained_model model = LlavaLlamaForCausalLM.from_pretrained( File "/home/dell/ENTER/envs/llava/lib/python3.10/site-packages/transformers/modeling_utils.py", line 3917, in from_pretrained dispatch_model(model, **device_map_kwargs) File "/home/dell/ENTER/envs/llava/lib/python3.10/site-packages/accelerate/big_modeling.py", line 364, in dispatch_model weights_map = OffloadedWeightsLoader( File "/home/dell/ENTER/envs/llava/lib/python3.10/site-packages/accelerate/utils/offload.py", line 150, in init raise ValueError("Need either a state_dict or a save_folder containing offloaded weights.") ValueError: Need either a state_dict or a save_folder containing offloaded weights.

[kdwivedi1985]

I am getting the same issue

ValueError Traceback (most recent call last) Cell In[5], line 1 ----> 1 pipeline = transformers.pipeline( 2 "text-generation", 3 model=model, 4 torch_dtype=torch.float16, 5 device_map="auto", 6 )

File /Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/transformers/pipelines/init.py:895, in pipeline(task, model, config, tokenizer, feature_extractor, image_processor, framework, revision, use_fast, token, device, device_map, torch_dtype, trust_remote_code, model_kwargs, pipeline_class, kwargs) 893 if isinstance(model, str) or framework is None: 894 model_classes = {"tf": targeted_task["tf"], "pt": targeted_task["pt"]} --> 895 framework, model = infer_framework_load_model( 896 model, 897 model_classes=model_classes, 898 config=config, 899 framework=framework, 900 task=task, 901 hub_kwargs, 902 **model_kwargs, 903 ) 905 model_config = model.config 906 hub_kwargs["_commit_hash"] = model.config._commit_hash ... raise ValueError("Need either a state_dict or a save_folder containing offloaded weights.") ValueError: Need either a state_dict or a save_folder containing offloaded weights.