fix: modify fallback of flash attention - Githubissues

DeepLink-org / DeepLinkExt

BSD 3-Clause "New" or "Revised" License

11 stars 0 forks source link

fix: modify fallback of flash attention #96

Closed POI-WX closed 5 months ago

POI-WX commented 5 months ago

Support gqa for the fallback of flash attention.
Add the corresponding test cases.
CI目前的7.0 软件栈无法通过使用casual mask测试，本地8.0软件栈可以测试通过。