Closed caissonfiv closed 3 months ago
I find it still slow, flash attn is already installed.
Look for these lines in your console:
Also the Gemma node should say (in console) if it's using "flash_attention_2" or "sdpa"
I find it still slow, flash attn is already installed.