Closed vgokhale closed 8 months ago
Added support for attention bias to flash attention. Bias is added to the result of the first GEMM (qk) before doing softmax.
Can you change the filename to reflect that bias is supported?
@zhanglx13 I renamed the file to just flash-attention.py and then added a description of what features are currently supported.
Added support for attention bias to flash attention. Bias is added to the result of the first GEMM (qk) before doing softmax.