search

xorbitsai / inference

Replace OpenAI GPT with another LLM in your app by changing a single line of code. Xinference gives you the freedom to use any LLM you need. With Xinference, you're empowered to run inference with any open-source language models, speech recognition models, and multimodal models, whether in the cloud, on-premises, or even on your laptop.
https://inference.readthedocs.io
Apache License 2.0
3.78k stars 322 forks source link

FEAT: support deepseek v2 chat and coder #1440

Open GXKIM opened 2 months ago

GXKIM commented 2 months ago

如题

qinxuye commented 2 months ago

deepseek coder 支持了吧

GXKIM commented 2 months ago

deepseek coder 支持了吧

v2 coder

adogcode commented 2 months ago

https://www.modelscope.cn/models/deepseek-ai/DeepSeek-V2-Chat/summary

100ZZ commented 1 month ago
image

Mac M系列这个问题能搞定么？我都是本地下载好的模型自定义注册 Model Family：deepseek-chat（other也一样） xinference：0.12.0

halexan commented 5 days ago

希望能增加deepseek v2 和 deepseek coder v2 的支持

vllm 0.5.1版本已支持deepseek v2, 详见vllm 0.5.1 release