Can you provide the inference version of DeepSeek based on vllm, deepspeed and tensorrt-llm

deepseek-ai / DeepSeek-MoE

DeepSeekMoE: Towards Ultimate Expert Specialization in Mixture-of-Experts Language Models

MIT License

982 stars 48 forks source link

Closed Eutenacity closed 7 months ago

Eutenacity commented 8 months ago

Can you provide the inference version of DeepSeek based on vllm, deepspeed and tensorrt-llm

zwd003 commented 7 months ago

we have support vllm in the latest version