Does flash-attention support RTX 4070ti?

Dao-AILab / flash-attention

Fast and memory-efficient exact attention

BSD 3-Clause "New" or "Revised" License

13.62k stars 1.25k forks source link

Does flash-attention support RTX 4070ti? #533

Open RuohuaTang opened 1 year ago

RuohuaTang commented 1 year ago

Hi there,

My Windows PC is with one RTX 4070ti. My cuda version is 11.8 and torch version is 2.0.0. I can not install most versions of flash-attn. I can only install flash-attn=1.0.4. But since I can not install flash-attn/csrc/layer_norm and flash-attn/csrc/rotary, I can not benefit from flash-attention.

Is there any method to install flash-attn, flash-attn layer_norm, and flash-attn rotary on my PC?

Thanks in advance!

tridao commented 1 year ago

I've no experience with Windows. It's only tested on Linux, so WSL would probably work.

Slipstream-Max commented 6 months ago

I‘m using RTX 4060ti 16g and download Flash attn.Most of it works just fine.Though some extensions can't install.(eg. layer_norm rotary)

tridao commented 6 months ago

4060 should also work.