Add MoE and pipelining support - Githubissues

mistralai / mistral-inference

Official inference library for Mistral models

https://mistral.ai/

Apache License 2.0

9.49k stars 835 forks source link

Add MoE and pipelining support #83

Closed diegolascasas closed 9 months ago

diegolascasas commented 9 months ago

Add MoE and Pipelining support
Update readme
Update requirements
Add faster loading
Make sliding window optional and add rope_theta with smart default