laekov / fastmoe

A fast MoE impl for PyTorch
https://fastmoe.ai
Apache License 2.0
1.52k stars 184 forks source link

[Feature] Make bias of gate optional for naive_gate and its subclasses. #185

Closed Zhang-RQ closed 8 months ago

Zhang-RQ commented 8 months ago

Some of public MoE weights don't need bias of gates. Make bias of gate optional can impove compatibility of newly public MoE weights.