Closed Zhang-RQ closed 8 months ago
Some of public MoE weights don't need bias of gates. Make bias of gate optional can impove compatibility of newly public MoE weights.
Some of public MoE weights don't need bias of gates. Make bias of gate optional can impove compatibility of newly public MoE weights.