Open JunxiChhen opened 2 months ago
Test tag is 0.5.3.post1-Gaudi-1.17.0
Thanks for the report! I've investigated this issue before and unfortunately, it's a bug in HCCL and is beyond vLLM's control, as the deadlock occurs after the main function exits. We are working with HCCL to resolve this.
While it's not ideal, you can exit by adding the following workaround after the benchmark:
import os
os._exit(0)
I will update this issue once we have a HCCL fix.
Testing possible fix #379
Your current environment
Command line:
🐛 Describe the bug
The script above cannot exit and hang there util using Ctrl+C to stop it manually.