SJTU-IPADS / PowerInfer

High-speed Large Language Model Serving on PCs with Consumer-grade GPUs
MIT License
7.95k stars 410 forks source link

Mixtral MoE support? #19

Open nivibilla opened 10 months ago

nivibilla commented 10 months ago

Prerequisites

Please answer the following questions for yourself before submitting an issue.

Feature Description

Support for Mixtral MoE

mathieu-heyarctic commented 10 months ago

https://github.com/SJTU-IPADS/PowerInfer/issues/13

BadisG commented 10 months ago

@mathieu-heyarctic https://github.com/SJTU-IPADS/PowerInfer/issues/13 is for Mistral, not Mixtral

dRAT3 commented 9 months ago

This would be OP