search

vllm-project / vllm

A high-throughput and memory-efficient inference and serving engine for LLMs
https://docs.vllm.ai
Apache License 2.0
22.1k stars 3.12k forks source link

[Bug]: 推理时异常 #5819

Open lvjianxin opened 1 week ago

lvjianxin commented 1 week ago

Your current environment

The output of `python collect_env.py`

🐛 Describe the bug

resp_list = inference_vllm(self.llm_engine, self.template, query_all)\n File \"/home/admin/miniconda3/lib/python3.10/site-packages/swift/llm/utils/vllm_utils.py\", line 298, in inference_vllm\n step_outputs = llm_engine.step()\n File \"/home/admin/miniconda3/lib/python3.10/site-packages/vllm/engine/llm_engine.py\", line 853, in step\n return self._process_model_outputs(output, scheduler_outputs)\n File \"/home/admin/miniconda3/lib/python3.10/site-packages/vllm/engine/llm_engine.py\", line 756, in _process_model_outputs\n self._process_sequence_group_outputs(seq_group, outputs)\n File \"/home/admin/miniconda3/lib/python3.10/site-packages/vllm/engine/llm_engine.py\", line 594, in _process_sequence_group_outputs\n parent_child_dict[sample.parent_seq_id].append(sample)\nKeyError: 1597

DarkLight1337 commented 1 week ago

Please provide details about the code which you used to trigger the error.