search

InternLM / InternEvo

InternEvo is an open-sourced lightweight training framework aims to support model pre-training without the need for extensive dependencies.
https://internevo.readthedocs.io/zh-cn/latest/?badge=latest
Apache License 2.0
310 stars 52 forks source link

[Feature] MoE模型里稠密层和专家层zero和并行的解耦 #325

Open sunpengsdu opened 2 months ago

sunpengsdu commented 2 months ago

Describe the feature

MoE模型里稠密层和专家层zero和并行的解耦

Will you implement it?