Open pranjalst opened 5 days ago
@pranjalst It seems you are using not latest, 1.16.0 version of SynapseAI with vllm-0.6.3. Please use latest SynapseAI 1.18.0 and HabanaAI vllm fork with branch v1.18.0 (tag: v0.5.3.post1+Gaudi-1.18.0).
@pranjalst did you try latest version of SynapseAI and vllm-fork?
Your current environment
Docker Image and Execution Command Overview
Docker Image Built From:
This image is designed for running PyTorch applications on Habana devices. It includes the VLLM fork and necessary dependencies installed via
requirements-hpu.txt
. Key configurations such as enabling lazy collectives for HPU are set up in the Dockerfile.Execution Command:
Error Encountered:
This error occurs during the execution of the VLLM API server, indicating a potential deadlock issue related to resource management in the HPU environment. Further investigation is needed to resolve this issue for successful deployment.
Model Input Dumps
nan
🐛 Describe the bug
nan
Before submitting a new issue...