search

Victorwz / LLaVA-Llama-3

Reproduction of LLaVA-v1.5 based on Llama-3-8b LLM backbone.
https://huggingface.co/weizhiwang/LLaVA-Llama-3-8B
Apache License 2.0
42 stars 5 forks source link

pretrain llava1.6? #3

Open lightmatmul opened 2 months ago

lightmatmul commented 2 months ago

I saw a mention that this repo has been upgraded to llava 1.6, is it possible to pretrain a llava 1.6 based on llama 3 ?

Victorwz commented 1 month ago

I saw a mention that this repo has been upgraded to llava 1.6, is it possible to pretrain a llava 1.6 based on llama 3 ?

Hi, I am sorry for the late reply. Currently, it only supports the multi-resolution inference. I have not tried to reimplement the multi-resolution training. If you are interested in this, you can follow https://github.com/arielnlee/LLaVA-1.6-ft/tree/main to fine-tune LLaVA-v1.6. Welcome to submit a PR to my repo.