Flash Attention building

nktice / AMD-AI

AMD (Radeon GPU) ROCm based setup for popular AI tools on Ubuntu 24.04

Other

156 stars 10 forks source link

Flash Attention building #10

Open CubeTheThird opened 4 weeks ago

CubeTheThird commented 4 weeks ago

While I know it's not currently listed in the build instructions, I'm curious if there's been any success in getting Flash Attention to work. Some time ago I was able to build it successfully, but could not use it due to (what I believe) the need for Torch 2.3+. Now that newer versions are out, I can no longer find myself being able to build it successfully.

Has this been looked into?

nktice commented 4 weeks ago

Thus far I have been unable to get the new FA2 working... I filed a report, describing what I have done, and how far I got : https://github.com/ROCm/flash-attention/issues/73 I have written this on getting their pre-built code packages loaded : https://github.com/nktice/AMD-AI/blob/main/flash-attention.md Alas even that does not appear to function as one would expect.