[BUG] MiniCPM-Llama3-V 2.5 doesn't work for VLLM mentioned in README

[nqyy]

是否已有关于该错误的issue或讨论？ | Is there an existing issue / discussion for this?

• [X] 我已经搜索过已有的issues和讨论 | I have searched the existing issues / discussions

该问题是否在FAQ中有解答？ | Is there an existing answer for this in FAQ?

• [X] 我已经搜索过FAQ | I have searched FAQ

当前行为 | Current Behavior

I tried https://github.com/OpenBMB/vllm/blob/main/examples/minicpmv_example.py and changed the model to openbmb/MiniCPM-Llama3-V-2_5. And I got this error:

Traceback (most recent call last): File "/data/tmp/chi/vllm/examples/minicpmv_example.py", line 125, in model = MiniCPMV_VLLM() File "/data/tmp/chi/vllm/examples/minicpmv_example.py", line 82, in init self.llm = LLM( File "/data/tmp/chi/vllm/vllm/entrypoints/llm.py", line 118, in init self.llm_engine = LLMEngine.from_engine_args( File "/data/tmp/chi/vllm/vllm/engine/llm_engine.py", line 277, in from_engine_args engine = cls( File "/data/tmp/chi/vllm/vllm/engine/llm_engine.py", line 148, in init self.model_executor = executor_class( File "/data/tmp/chi/vllm/vllm/executor/executor_base.py", line 41, in init self._init_executor() File "/data/tmp/chi/vllm/vllm/executor/gpu_executor.py", line 22, in _init_executor self._init_non_spec_worker() File "/data/tmp/chi/vllm/vllm/executor/gpu_executor.py", line 51, in _init_non_spec_worker self.driver_worker.load_model() File "/data/tmp/chi/vllm/vllm/worker/worker.py", line 117, in load_model self.model_runner.load_model() File "/data/tmp/chi/vllm/vllm/worker/model_runner.py", line 162, in load_model self.model = get_model( File "/data/tmp/chi/vllm/vllm/model_executor/model_loader/init.py", line 19, in get_model return loader.load_model(model_config=model_config, File "/data/tmp/chi/vllm/vllm/model_executor/model_loader/loader.py", line 222, in load_model model = _initialize_model(model_config, self.load_config, File "/data/tmp/chi/vllm/vllm/model_executor/model_loader/loader.py", line 90, in _initialize_model return model_class(config=model_config.hf_config, File "/data/tmp/chi/vllm/vllm/model_executor/models/minicpmv.py", line 202, in init self.llm = MiniCPMForCausalLM(config, linear_method) File "/data/tmp/chi/vllm/vllm/model_executor/models/minicpm.py", line 438, in init self.scale_width = self.config.hidden_size / self.config.dim_model_base File "/data/tmp/chi/vllm/venv/lib/python3.10/site-packages/transformers/configuration_utils.py", line 264, in getattribute return super().getattribute(key) AttributeError: 'MiniCPMVConfig' object has no attribute 'dim_model_base'

期望行为 | Expected Behavior

No response

复现方法 | Steps To Reproduce

No response

运行环境 | Environment

- OS:
- Python:
- Transformers:
- PyTorch:
- CUDA (`python -c 'import torch; print(torch.version.cuda)'`):

备注 | Anything else?

No response

[iceflame89]

We are working on the feature of deploy MiniCPM-Llama3-V 2.5 with vllm, please stay tuned.

[kennyLSN]

We are working on the feature of deploy MiniCPM-Llama3-V 2.5 with vllm, please stay tuned.

同样出现这个问题，请问现在有优化进展了吗

[winsonxxxx]

同样出现这个问题

[weiminw]

是否有进展了. 目前不用vllm 是否可以支持并发推理?