Closed zero0kiriyu closed 2 years ago
Thank you for your suggestion. For the first problem, i think that in causal attention, src_len and tgt_len must be same, this can be verified in the following code: https://github.com/idiap/fast-transformers/blob/master/fast_transformers/causal_product/causal_product_cpu.cpp#L79
For the second problem, we have test EPFL's causal linear attention, but it does not seem to be faster, so for now use the torch version.
Thank you for your great work! I am currently working on a seq2seq task and I found the causal attention code only works the src_len and the tgt_len are the same. Also, I suggest that you could adopt EPFL's causal linear attention CUDA code to improve the speed of causal attention.