issues
search
Rishit-dagli
/
Nystromformer
An implementation of the Nyströmformer, using Nystrom method to approximate standard self attention
Apache License 2.0
55
stars
4
forks
source link
Implement Nyström Attention
#2
Closed
Rishit-dagli
closed
2 years ago
Rishit-dagli
commented
2 years ago
Refresher:
https://www.cs.tau.ac.il/~amir1/PS/Subsampling.pdf
https://arxiv.org/pdf/1408.2044
https://www.newton.ac.uk/files/seminar/20080626113012301-151739.pdf
Reference Material:
https://arxiv.org/abs/2102.03902
https://www.youtube.com/watch?v=m-zrcmRd7E4
https://proceedings.mlr.press/v37/lima15.html
Refresher:
Reference Material: