Traceback (most recent call last):
File "", line 1, in
File "/usr/local/lib/python3.10/dist-packages/torch/cuda/memory.py", line 685, in mem_get_info
return torch.cuda.cudart().cudaMemGetInfo(device)
RuntimeError: HIP error: invalid argument
HIP kernel errors might be asynchronously reported at some other API call, so the stacktrace below might be incorrect.
For debugging consider passing AMD_SERIALIZE_KERNEL=3
Compile with TORCH_USE_HIP_DSA to enable device-side assertions.
"""
Your current environment
🐛 Describe the bug
I used this docker : https://hub.docker.com/r/eliovp/rocm6.1.2_py3.10_torch2.5_vllm0.5_bkc with MI100 .
But I always met this: """
Traceback (most recent call last): File "", line 1, in
File "/usr/local/lib/python3.10/dist-packages/torch/cuda/memory.py", line 685, in mem_get_info
return torch.cuda.cudart().cudaMemGetInfo(device)
RuntimeError: HIP error: invalid argument
HIP kernel errors might be asynchronously reported at some other API call, so the stacktrace below might be incorrect.
For debugging consider passing AMD_SERIALIZE_KERNEL=3
Compile with
TORCH_USE_HIP_DSA
to enable device-side assertions. """