search

X-PLUG / mPLUG-Owl

mPLUG-Owl: The Powerful Multi-modal Large Language Model Family
https://www.modelscope.cn/studios/damo/mPLUG-Owl
MIT License
2.25k stars 171 forks source link

Have you tuned the vision model in sft stage? #179

Closed xmy0916 closed 10 months ago

xmy0916 commented 10 months ago

I found code here: https://github.com/X-PLUG/mPLUG-Owl/blob/main/mPLUG-Owl2/scripts/finetune.sh#L32

you have freezed the vision backbone, but according to your paper, it seems all the parrams have been trained? image

MAGAer13 commented 10 months ago

Yes, this is just reimplementation of our model. You can enable it during training and you should implement layer-wise learning rate by yourself.

xmy0916 commented 10 months ago

Yes, this is just reimplementation of our model. You can enable it during training and you should implement layer-wise learning rate by yourself.

Thanks for your great job and answer~