Open Yuki-Kokomi opened 1 week ago
This just happened to me also. After the pre-training phase I am trying to perform inference however seems there is a miss-match between the saved model on /<model_path>/language_model/pytorch_model.bin
A quick work around for now would be to follow the solkution made on the issue https://github.com/vllm-project/vllm/issues/3323 linked by @Yuki-Kokomi, by copying the embed_token
weight onto the lm_head
:
language_model_ckp_path = os.path.join(model_name_or_path, 'language_model/pytorch_model.bin')
language_model_ckp = load_base_ckp_for_lora(language_model_ckp_path)
# This line is what does the trick
language_model_ckp['lm_head.weight'] = language_model_ckp['model.embed_tokens.weight']
model.language_model.load_state_dict(language_model_ckp)
However, I am not able to qualitatively validate that this fix works fine.
Thank you @ggcr for the solution. It works perfectly.
I can provide a PR by doing this checking if the LLM backbone used is Gemma if the authors want.
I can provide a PR by doing this checking if the LLM backbone used is Gemma if the authors want.
Hi, we do encourage you to initiate a PR!
When attempting to merge LoRA weights into the TinyLLaVA-Gemma-SigLIP-2.4B model, I encountered a RuntimeError due to a missing key lm_head.weight in the GemmaForCausalLM state_dict. The specific error traceback is as follows:
This issue seems similar to vllm-project/vllm#3323. Any insights or solutions to resolve the missing "lm_head.weight" key would be greatly appreciated.