Open hit56 opened 5 months ago
I'm signing on to this one. I need to run inference on T4 gpus..
+1
OpenAI's Triton implementation of flash attention works on Turing GPUs (just tested this myself):
https://github.com/openai/triton/blob/main/python/tutorials/06-fused-attention.py
@rationalism FlashAttention1.X works for Turing gpus, the problem is with version 2 or newer. please check the following line, it seems like it is not supported by Turing GPUs (or it is a typo)
@AvivSham I saw that, but it does in fact seem to work on my machine
Yes, please support Tesla T4!
really need turing support, is there a way we can do this on our own /sigh