search

WisconsinAIVision / ViP-LLaVA

[CVPR2024] ViP-LLaVA: Making Large Multimodal Models Understand Arbitrary Visual Prompts
https://vip-llava.github.io/
Apache License 2.0
282 stars 21 forks source link

[Usage] Pretrain stage did not save all required files #29

Open james20141606 opened 1 month ago

james20141606 commented 1 month ago

Describe the issue

Hi, I am trying to perform the pretrain task and the files which are saved are:

Screenshot 2024-08-25 at 11 54 49 PM

and within each checkpoint folder there are only two files config.json mm_projector.bin

It seems many files are missing, compared to files shown here https://huggingface.co/mucai/vip-llava-7b/tree/main

The missing files make me impossible to launch the finetuning stage. Could you provide some guidance on saving all the files? Or does it make sense to reuse the model weights in https://huggingface.co/mucai/vip-llava-7b/tree/main I am confused if it makes sense to only use my updated mm_projector.bin and still use the files you provided?

mu-cai commented 1 week ago

During pretraining stage, we do not train LLM, which is explained in the paper and docs. Thanks!