[Feature] 请问可以支持智谱团队的CogVLM2的量化嘛？

InternLM / lmdeploy

LMDeploy is a toolkit for compressing, deploying, and serving LLMs.

https://lmdeploy.readthedocs.io/en/latest/

Apache License 2.0

3.15k stars 281 forks source link

Open EasonGZY opened 2 weeks ago

EasonGZY commented 2 weeks ago

请问可以支持智谱团队的CogVLM2的量化嘛？

No response

No response

lvhan028 commented 2 weeks ago

目前还没有规划这个。得等我们把 turbomind 的量化推理 kernel 移植到 pytorch engine 后，才能再考虑 CogVLM2 的量化了