Closed zollty closed 4 months ago
感谢反馈,当前0.3.3版本没问题,0.4版本后续更新支持
File "/usr/local/lib/python3.10/dist-packages/vllm/model_executor/models/zhinao.py", line 31, in from vllm.model_executor.input_metadata import InputMetadata ModuleNotFoundError: No module named 'vllm.model_executor.input_metadata'
maybe should change
from vllm.model_executor.input_metadata import InputMetadata
to
from vllm.attention import Attention, AttentionMetadata
Please use zhinao_040.py file in vllm 4.0
File "/usr/local/lib/python3.10/dist-packages/vllm/model_executor/models/zhinao.py", line 31, in
from vllm.model_executor.input_metadata import InputMetadata
ModuleNotFoundError: No module named 'vllm.model_executor.input_metadata'
maybe should change
from vllm.model_executor.input_metadata import InputMetadata
tofrom vllm.attention import Attention, AttentionMetadata