[Bug] 尚未支持Qwen2-VL系列，与ReadMe不符合~

xuexidi commented 1 month ago

Checklist

[X] 1. I have searched related issues but cannot get the expected help.
[X] 2. The bug has not been fixed in the latest version.
[X] 3. Please note that if the bug-related issue you submitted lacks corresponding environment info and a minimal reproducible demo, it will be challenging for us to reproduce and resolve the issue, reducing the likelihood of receiving feedback.

Describe the bug

您好，非常感谢您的开源工作，我看ReadMe写已支持Qwen2-VL-2B、Qwen2-VL-7B，但是我用指令启动的时候报错了：

2024-09-25 10:24:26,434 - lmdeploy - WARNING - Try to run with pytorch engine because `/home/xue/model_weights/Qwen2-VL/Qwen2-VL-7B-Instruct/` is not explicitly supported by lmdeploy.

.....
RuntimeError: Can not found rewrite for architectures: ['Qwen2VLForConditionalGeneration']

Reproduction

lmdeploy serve api_server /home/xue/model_weights/Qwen2-VL/Qwen2-VL-7B-Instruct/ --server-name 0.0.0.0 --server-port 8080--tp 1 --model-name Qwen2-7B-Instruct

Environment

lmdeploy==0.6.0

Error traceback

No response

lvhan028 commented 1 month ago

The latest main branch DOES support qwen2-vl. It hasn't been released yet.

xingenju commented 1 month ago

Hi @lvhan028, When can be released?

lvhan028 commented 1 month ago

This week

thesby commented 1 month ago

希望能支持 qwen2 vl 72B，谢谢

douyh commented 1 month ago

 ValueError: The checkpoint you are trying to load has model type `qwen2_vl` but Transformers does not recognize this architecture. This could be because of an issue with the checkpoint, or because your version of Transformers is out of date.

I run qwen2-vl-2b-instruct-awq with lmdeploy 0.6.0 and get an error.

Will it be solved by the coming new version?

lvhan028 commented 1 month ago

Yes. It will be released on v0.6.1

InternLM / lmdeploy