search

mistralai / mistral-inference

Official inference library for Mistral models
https://mistral.ai/
Apache License 2.0
9.16k stars 804 forks source link

speed up inference? #169

Open xxyp opened 1 month ago

xxyp commented 1 month ago

employing Codestral-22B-v0.1 by Mistral-inference, the speed seems slow. there any methods to speed up the inference ?