Trainable parameters during finetuning for medical VQA

microsoft / LLaVA-Med

Large Language-and-Vision Assistant for Biomedicine, built towards multimodal GPT-4 level capabilities.

Other

1.59k stars 202 forks source link

Open DopamineLcy opened 5 months ago

DopamineLcy commented 5 months ago

Thank you for your impressive work!

I'm wondering about the image encoder, projection layer, and LM, which are trainable during finetuning for medical VQA.

Looking forward to your reply.

Best,

thedaffodil commented 4 months ago

did you get the answer? can we decide which parameters to train while fine-tuning this model?