search

PKU-YuanGroup / MoE-LLaVA

Mixture-of-Experts for Large Vision-Language Models
https://arxiv.org/abs/2401.15947
Apache License 2.0
1.9k stars 121 forks source link

[Question] What exactly does the language model mean? #84

Open dana-niu opened 2 months ago

dana-niu commented 2 months ago

Question

Hello, I want to ask The language model in LLaVA-1.5 is vicuna. Do you mean FFN AND Self-Attention layers by language model?