请问什么时候会支持对CogVLM2的量化

InternLM / lmdeploy

LMDeploy is a toolkit for compressing, deploying, and serving LLMs.

https://lmdeploy.readthedocs.io/en/latest/

Apache License 2.0

4.56k stars 409 forks source link

请问什么时候会支持对CogVLM2的量化 #1902

Open EasonGZY opened 4 months ago

EasonGZY commented 4 months ago

您好，请问什么时候会支持对CogVLM2的量化，模型来自zhipu的https://huggingface.co/THUDM/cogvlm2-llama3-chat-19B，能否用https://github.com/InternLM/lmdeploy/blob/main/docs/en/quantization/w4a16.md来辅助进行量化，谢谢！

lvhan028 commented 4 months ago

0.5.0 has supported cogvlm2. May give it a try.

lvhan028 commented 3 months ago

@AllentDan @grimoire

lvhan028 commented 3 months ago

0.5.0 has supported cogvlm2. May give it a try.

My mistake. 0.5.0 support cogvlm2 but hasn't support its quantization yet.