Open AllentDan opened 5 months ago
LMDeploy, as an AI deployment platform supporting multiple backend services, has always been committed to providing fast and stable AI model deployment services.
Now, it supports accelerating the inference and service of DeepSeek VL models. Check https://github.com/InternLM/lmdeploy/pull/1335
请问DeepSeek-VL-chat-7B如果使用LMDploy加速,和DeepSeek-VL-chat-1.3B相比,推理速度如何呢?
DeepSeek-VL-chat-1.3B也可以支持量化么?
LMDeploy, as an AI deployment platform supporting multiple backend services, has always been committed to providing fast and stable AI model deployment services.
Now, it supports accelerating the inference and service of DeepSeek VL models. Check https://github.com/InternLM/lmdeploy/pull/1335