search

ddw2AIGROUP2CQUPT / PA-LLaVA

26 stars 1 forks source link

Request open source the projector weights #5

Open lsnls opened 2 weeks ago

lsnls commented 2 weeks ago

Hello, authors!

Thanks for your valuable work! I am trying to run zero_shot.py on my own dataset using the PA-LLaVA checkpoint. Could you please clarify which checkpoint the --llava ./instruction_tuning_weight_ft parameter corresponds to? I have tried all the open-sourced weights on Huggingface, but none match. Can you also open source the projector weight?

Thanks for your help and response!

Best regards, Suoni

lsnls commented 2 weeks ago

@ddw2AIGROUP2CQUPT

ddw2AIGROUP2CQUPT commented 2 weeks ago

Thank you for your interest. In fact, we have already released the weights and provided instructions in the README.md. Based on your question, it seems that you need to merge the weights into the required format. The instructions for doing so are available on Hugging Face.

lsnls commented 1 week ago

Hello, @ddw2AIGROUP2CQUPT , thank you so much for your quick response!

Following the instructions on Hugging Face, I ran the command xtuner convert pth_to_hf pallava_instruction_tuning.py path_to/instruction_tuning_weight_ft.pth path_to/instruction_tuning_weight_ft. However, I encountered an error: TypeError: LLaVAModel.__init__() got an unexpected keyword argument 'projector_type'. Could you please assist me in resolving this issue?

Thank you again!