[Feature]: Not support Qwen-VL-Chat

[NEOOOOOOOOOO]

🚀 The feature, motivation and pitch

While trying to execute the following code:

import requests
url = "http://host:port/v1/chat/completions"

payload = json.dumps(
    {
        "model": "Qwen-VL-Chat",
        "temperature": 0.1,
        "messages": [
            {
                "role": "user",
                "content": [
                    {"type": "text", "text": "what's this?"},
                    {"type": "image_url", "image_url": {"url": "https://dashscope.oss-cn-beijing.aliyuncs.com/images/dog_and_girl.jpeg"}},

                ],
            }
        ],
    }
)
headers = {"Content-Type": "application/json", "Authorization": "Bearer sk-token"}

response = requests.request("POST", url, headers=headers, data=payload)

I encountered the following error message:

{"error":{"message":"Unknown model type: {model_type} (request id: 2024080114362169993116371613991)","type":"upstream_error","param":"400","code":"bad_response_status_code"}}

vllm Version： 0.5.3.post

It appears that the Qwen-VL-Chat model is not supported by the vllm API. Can you please assist in resolving this issue?

Thank you!

Alternatives

No response

Additional context

No response

[ywang96]

Qwen-VL is not supported by vLLM yet but you're very welcome to submit a PR to do so. Please see all supported vision language models here https://docs.vllm.ai/en/latest/models/supported_models.html#vision-language-models

[alex-jw-brooks]

I am looking into adding support for image inputs for Qwen-VL/Qwen-VL-Chat! Definitely open to collaborating somehow though 😄

[zhujinhua]

Hi, is anything update about supporting Qwen-VL/Qwen-VL-Chat in vllm?

[alex-jw-brooks]

Hello @zhujinhua! I'm still working on it and have made some progress, I am hoping to have a PR up for it soon 🤞

[hmellor]

Closing as duplicate of #962