Support of AMD consumer GPUs

vllm-project / vllm

A high-throughput and memory-efficient inference and serving engine for LLMs

https://docs.vllm.ai

Apache License 2.0

22.1k stars 3.12k forks source link

Support of AMD consumer GPUs #3040

Open arno4000 opened 4 months ago

arno4000 commented 4 months ago

Is there a technical reason, why only AMD MI200 GPUs or newer are supported? I get the error RuntimeError: FlashAttention only supports AMD MI200 GPUs or newer. when I try to run vllm on my RX 7900 XTX.

Wintoplay commented 2 months ago

what version of rocm?

ihejia commented 1 month ago

I also encountered the same problem, my software version is ROCm6.0,and my graphics card is 7900XT

papasanimohansrinivas commented 3 days ago

I also encountered the same problem, my software version is ROCm6.0,and my graphics card is 7900XT

same here