search

Qihoo360 / 360zhinao

360zhinao
Apache License 2.0
274 stars 23 forks source link

Not compatible with vllm 4.0 #3

Closed zollty closed 4 months ago

zollty commented 4 months ago

File "/usr/local/lib/python3.10/dist-packages/vllm/model_executor/models/zhinao.py", line 31, in from vllm.model_executor.input_metadata import InputMetadata ModuleNotFoundError: No module named 'vllm.model_executor.input_metadata'

maybe should change from vllm.model_executor.input_metadata import InputMetadata to

from vllm.attention import Attention, AttentionMetadata

gongxiaochun commented 4 months ago

感谢反馈，当前0.3.3版本没问题，0.4版本后续更新支持

garycaokai commented 4 months ago

File "/usr/local/lib/python3.10/dist-packages/vllm/model_executor/models/zhinao.py", line 31, in from vllm.model_executor.input_metadata import InputMetadata ModuleNotFoundError: No module named 'vllm.model_executor.input_metadata'

maybe should change from vllm.model_executor.input_metadata import InputMetadata to

from vllm.attention import Attention, AttentionMetadata

Please use zhinao_040.py file in vllm 4.0