Is there a flash attention whl for win11?

kijai / ComfyUI-MochiWrapper

Apache License 2.0

638 stars 53 forks source link

Is there a flash attention whl for win11? #54

Open bbecausereasonss opened 2 weeks ago

bbecausereasonss commented 2 weeks ago

See above.

kijai commented 2 weeks ago

Flash attention isn't really useful anymore, sdpa works fine and sage_attn, if it's supported by your hardware and you can install it, is a lot faster.

bbecausereasonss commented 2 weeks ago

Looks like Win11 is shit out of luck. Maybe I should re-install WSL.

Wladastic commented 2 weeks ago

You can use flash attention on WSL, but building the whl is hit or miss. Had to reinstall torch and cuda multiple times for it to work. Also check if your Nvidia Driver includes the same cuda or above of what you install in wsl2.