search

PKU-YuanGroup / MoE-LLaVA

Mixture-of-Experts for Large Vision-Language Models
https://arxiv.org/abs/2401.15947
Apache License 2.0
1.9k stars 121 forks source link

[Question] 论文参数讨论 #67

Open bufanx opened 5 months ago

bufanx commented 5 months ago

Question

在论文3.4节的Auto-Regressive Loss中，请问K指代的是否为visual tokens + textual tokens的序列长度，如果是这样的话，K是否应该为P+N？

puppy2000 commented 5 months ago

我也觉得应该是P+N