moe模型的适配问题

wenet-e2e / wenet

Production First and Production Ready End-to-End Speech Recognition Toolkit

https://wenet-e2e.github.io/wenet/

Apache License 2.0

3.87k stars 1.03k forks source link

Closed programYoung closed 15 hours ago

programYoung commented 2 weeks ago

请问一下目前moe模型只能用pt模型测试吗，能否在runtime上运行；moe模型的转onnx是否适配；在大数据集上训练的1B的moe模型训练速度能到多少

xingchensong commented 15 hours ago

只能pt；libtorch的runtime可以无缝跑；转onnx因为有分支操作，所以需要适配；速度见paper https://arxiv.org/pdf/2404.16407