feat(kernel): Add a pytorch bind for flashattention.

TiledTensor / TiledCUDA

TiledCUDA is a highly efficient kernel template library designed to elevate CUDA C’s level of abstraction for processing tiles.

MIT License

157 stars 10 forks source link

feat(kernel): Add a pytorch bind for flashattention. #128

Closed KuangjuX closed 2 months ago

KuangjuX commented 2 months ago

Currently I have added the binding of the flashattention kernel on PyTorch, however the results obtained are not as expected. I will check the implementation of flashattention and obtain the correct results before initiating the review.