search

ROCm / triton

Development repository for the Triton language and compiler
MIT License
83 stars 27 forks source link

Add support for ALiBi-style attention bias #417

Closed vgokhale closed 8 months ago

vgokhale commented 9 months ago

Added support for attention bias to flash attention. Bias is added to the result of the first GEMM (qk) before doing softmax.

zhanglx13 commented 9 months ago

Can you change the filename to reflect that bias is supported?

vgokhale commented 8 months ago

@zhanglx13 I renamed the file to just flash-attention.py and then added a description of what features are currently supported.