[Usage] Error when use cli.py after merge-lora-weights.py

Describe the issue

Issue: I used finetune_lora.sh to finetune vicuna-v1.5-13b with custom data. After it, I got a folder with adapter_model.safetensors, non_lora_trainables.bin. Then I merged it by merge-lora-weights.py, and I got another folder with three pytorch-model.bin. When I used cli.py, it occured an error.

I didn't use the --load-4bit because of this issue #744. I met the same error before, so I removed it. Command:

CUDA_VISIBLE_DEVICES=0 python -m llava.serve.cli --model-path /home/xhw/LLaVA/llava-v1.5-13b-lora-merge --model-base /home/xhw/LLaVA/vicuna-13b --image-file /home/xhw/LLaVA/images/8.jpg

Log:

[2024-07-25 06:06:17,295] [INFO] [real_accelerator.py:161:get_accelerator] Setting ds_accelerator to cuda (auto detect)
Loading LLaVA from base model...
Loading checkpoint shards: 100%|███████████████████████████████████████████████████████████████████████████████| 3/3 [00:13<00:00,  4.42s/it]
Some weights of LlavaLlamaForCausalLM were not initialized from the model checkpoint at /home/xhw/LLaVA/vicuna-13b and are newly initialized: ['model.mm_projector.0.bias', 'model.mm_projector.0.weight', 'model.mm_projector.2.bias', 'model.mm_projector.2.weight']
You should probably TRAIN this model on a down-stream task to be able to use it for predictions and inference.
Loading additional LLaVA weights...
Loading LoRA weights...
Traceback (most recent call last):
  File "/home/xhw/anaconda3/envs/llava/lib/python3.10/runpy.py", line 196, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/home/xhw/anaconda3/envs/llava/lib/python3.10/runpy.py", line 86, in _run_code
    exec(code, run_globals)
  File "/home/xhw/LLaVA/llava/serve/cli.py", line 126, in <module>
    main(args)
  File "/home/xhw/LLaVA/llava/serve/cli.py", line 32, in main
    tokenizer, model, image_processor, context_len = load_pretrained_model(args.model_path, args.model_base, model_name, args.load_8bit, args.load_4bit, device=args.device)
  File "/home/xhw/LLaVA/llava/model/builder.py", line 83, in load_pretrained_model
    model = PeftModel.from_pretrained(model, model_path)
  File "/home/xhw/anaconda3/envs/llava/lib/python3.10/site-packages/peft/peft_model.py", line 430, in from_pretrained
    model.load_adapter(model_id, adapter_name, is_trainable=is_trainable, **kwargs)
  File "/home/xhw/anaconda3/envs/llava/lib/python3.10/site-packages/peft/peft_model.py", line 984, in load_adapter
    adapters_weights = load_peft_weights(model_id, device=torch_device, **hf_hub_download_kwargs)
  File "/home/xhw/anaconda3/envs/llava/lib/python3.10/site-packages/peft/utils/save_and_load.py", line 415, in load_peft_weights
    has_remote_safetensors_file = file_exists(
  File "/home/xhw/anaconda3/envs/llava/lib/python3.10/site-packages/huggingface_hub/utils/_validators.py", line 106, in _inner_fn
    validate_repo_id(arg_value)
  File "/home/xhw/anaconda3/envs/llava/lib/python3.10/site-packages/huggingface_hub/utils/_validators.py", line 154, in validate_repo_id
    raise HFValidationError(
huggingface_hub.errors.HFValidationError: Repo id must be in the form 'repo_name' or 'namespace/repo_name': '/home/xhw/LLaVA/llava-v1.5-13b-lora-merge'. Use `repo_type` argument if needed.

Here is my folder screenshots. Screenshots:

haotian-liu / LLaVA

[Usage] Error when use cli.py after merge-lora-weights.py #1629

Describe the issue