search

QwenLM / Qwen2-VL

Qwen2-VL is the multimodal large language model series developed by Qwen team, Alibaba Cloud.
Apache License 2.0
2.04k stars 116 forks source link

使用vllm推理Qwen2-VL-2B-Instruct-GPTQ-Int4报错 #123

Open jingk-1992 opened 2 weeks ago

jingk-1992 commented 2 weeks ago

使用vllm推理Qwen2-VL-2B-Instruct-GPTQ-Int4报错，一直提示： File "/usr/local/venv/model_llm/lib/python3.10/site-packages/vllm/entrypoints/openai/api_server.py", line 130, in build_async_engine_client_from_engine_args if (model_is_embedding(engine_args.model, engine_args.trust_remote_code, File "/usr/local/venv/model_llm/lib/python3.10/site-packages/vllm/entrypoints/openai/api_server.py", line 71, in model_is_embedding return ModelConfig(model=model_name, File "/usr/local/venv/model_llm/lib/python3.10/site-packages/vllm/config.py", line 222, in init self.max_model_len = _get_and_verify_max_len( File "/usr/local/venv/model_llm/lib/python3.10/site-packages/vllm/config.py", line 1739, in _get_and_verify_max_len assert "factor" in rope_scaling AssertionError

D4YON3 commented 1 week ago

same

fyabc commented 1 week ago

@jingk-1992 @D4YON3 您好，目前Qwen2-VL的支持尚未合并到vllm官方代码中，您可参考此处链接，从源码安装vllm或使用我们提供的docker镜像。

jingk-1992 commented 1 week ago

谢谢！

fyabc commented 1 week ago

@jingk-1992 @D4YON3 此外，目前最新版本的transformers有一个已知bug也会导致这个问题，请参考此处的提示，使用pip install git+https://github.com/huggingface/transformers@21fac7abba2a37fae86106f87fcf9974fd1e3830安装正确版本的transformers，或参考这里的临时方法。