search

SkunkworksAI / BakLLaVA

Apache License 2.0
680 stars 46 forks source link

Inference error: size mismatch for model.mm_projector.weight #19

Open gameveloster opened 5 months ago

gameveloster commented 5 months ago

After doing a lora finetuning on the pretrained model using finetune_lora.sh, I tried to do inference using CLI

python -m llava.serve.cli \
    --model-path ./checkpoints/llava-Mistral-7B-v0.1-finetune_lora_1 \
    --model-base /models/llm/Mistral-7B-v0.1 \
    --image-file "https://llava-vl.github.io/static/images/view.jpg"

and received this error when loading non_lora_trainables.bin into the model

RuntimeError: Error(s) in loading state_dict for LlavaMistralForCausalLM:
    size mismatch for model.mm_projector.weight: copying a param with shape torch.Size([4096, 1024]) from checkpoint, the shape in current model is torch.Size([2097152, 1]).

There is no problem loading the Bakkala-1 checkpoint by running

python -m llava.serve.cli \
    --model-path /models/lmm/BakLLaVA-1 \
    --image-file "https://llava-vl.github.io/static/images/view.jpg"

Can I have some help to solve this crash when trying to use the LoRA created from the finetuning please? Thank you

zhyhome commented 3 months ago

How did you solve this problem?