search

dvlab-research / MGM

Official repo for "Mini-Gemini: Mining the Potential of Multi-modality Vision Language Models"
Apache License 2.0
3.21k stars 278 forks source link

lora initialisation missing from builder.py #106

Open adrielkuek opened 6 months ago

adrielkuek commented 6 months ago

Hi, currently peft is supported in train.py with lora_enable flag, however during inference when loading model.builder, there seems to be a mismatch if the model was trained with lora or qlora. Wondering whether the team would be updating builder code base from LlaVA to include peft support?

yanwei-li commented 6 months ago

Hi, all our models are fully finetunded. Actually, I did not try LoRA. This could be checked and supported soon.