(Question) How to set up MOE?

guinmoon / LLMFarm

llama and other large language models on iOS and MacOS offline using GGML library.

https://llmfarm.site

MIT License

1.05k stars 62 forks source link

(Question) How to set up MOE? #73

Open MiniPhantom opened 2 weeks ago

MiniPhantom commented 2 weeks ago

Is mixture of experts just limited to Mixtral, or can smaller models such as Qwen’s moe be run too? If so how to set it up, as there are no visible options.

Anyways thanks for your amazing work!