search

Victorwz / LLaVA-Llama-3

Reproduction of LLaVA-v1.5 based on Llama-3-8b LLM backbone.
https://huggingface.co/weizhiwang/LLaVA-Llama-3-8B
Apache License 2.0
53 stars 7 forks source link

Compatible with LoRA finetune #2

Closed Sprinter1999 closed 4 months ago

Sprinter1999 commented 5 months ago

Hi authors, thank you a lot for providing this great repo for our community. It seems this repo is based on Haotianliu's LLaVA's original repo and support LLaMA-3 as the LLM part of the VLM. I wonder if it is possible to refer to the configuration of finetune.sh and modify the original finetune_lora.sh in Haotian's repo to support the LoRA finetune of LLaVA-llama-3?

Best regards, Xuefeng

chiuwhsin commented 4 months ago

Hello @Sprinter1999 , Have you resolved this issue? I also would like to know if it is possible to refer to the configuration of finetune.sh and modify the original finetune_lora.sh in Haotian's repository to support the LoRA finetuning of LLaVA-llama-3.

Your response in this matter would be greatly appreciated!

Victorwz commented 4 months ago

Hello @Sprinter1999 , Have you resolved this issue? I also would like to know if it is possible to refer to the configuration of finetune.sh and modify the original finetune_lora.sh in Haotian's repository to support the LoRA finetuning of LLaVA-llama-3.

Your response in this matter would be greatly appreciated!

Hi, I upgrade the codebase to llava1.6 and also add the lora fine-tuning script. You can try and discuss that with me.