[Bug]: Assert Error: len(seqs) == 1

efsotr commented 2 days ago

Your current environment

vllm == 0.6.2

Model Input Dumps

No response

🐛 Describe the bug

/home/linli/anaconda3/envs/LLM/lib/python3.11/site-packages/vllm/core/scheduler.py line 1247 assert len(seqs) == 1

Before submitting a new issue...

[X] Make sure you already searched for relevant issues, and asked the chatbot living at the bottom right corner of the documentation page, which can answer lots of frequently asked questions.

DarkLight1337 commented 2 days ago

Please provide more information about how you triggered this error.

efsotr commented 2 days ago

Error in child process: Traceback info: Traceback (most recent call last): File "/home/linli/PO_clean/llmtrainer/multi_process.py", line 15, in worker_func return process_func(inputs, worker_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/linli/PO_clean/llmtrainer/get_output_vllm.py", line 181, in process step_outputs = vllm_engine.step() ^^^^^^^^^^^^^^^^^^ File "/home/linli/anaconda3/envs/LLM/lib/python3.11/site-packages/vllm/engine/llm_engine.py", line 1217, in step ) = self.scheduler[virtual_engine].schedule() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/linli/anaconda3/envs/LLM/lib/python3.11/site-packages/vllm/core/scheduler.py", line 1247, in schedule assert len(seqs) == 1 ^^^^^^^^^^^^^^ AssertionError

efsotr commented 2 days ago

https://gist.github.com/efsotr/e89e5ba418b02931f6500bf71fa7282f

model=./Meta-Llama-3-70B-Instruct
dir=train_data_llama3-70b
export CUDA_VISIBLE_DEVICES=0,1,2,3,4,5,6,7 
nohup python get_output_vllm.py  -sp ./eval/gen_5r_t0.8.json -o data/$dir --model $model -td ./data/train_data -tf train.json.gz -tp 4 -pt chat

LittleCoder12345 commented 2 days ago

the same error at #8844 and waiting for a solution

vllm-project / vllm