search

PKU-YuanGroup / MoE-LLaVA

Mixture-of-Experts for Large Vision-Language Models
https://arxiv.org/abs/2401.15947
Apache License 2.0
1.9k stars 121 forks source link

Moe finetuning error #77

Open sahilqure opened 4 months ago

sahilqure commented 4 months ago

Getting this error while finetuning the moe: AttributeError: 'MoELLaVA MistralModel' object has no attribute '_prepare_decoder_attention_mask'