Closed XFeiF closed 3 years ago
Paper Code-pytorch
Authors: Iz Beltagy, Matthew E. Peters, Arman Cohan
Youtube Explanation by Yannic Kilcher.
The Longformer extends the Transformer by introducing sliding window attention and sparse global attention. This allows for the processing of much longer documents than classic models like BERT.
XFeiF
commented
3 years ago 
Paper
Code-pytorch
Authors:
Iz Beltagy, Matthew E. Peters, Arman Cohan
Youtube Explanation by Yannic Kilcher.
The Longformer extends the Transformer by introducing sliding window attention and sparse global attention. This allows for the processing of much longer documents than classic models like BERT.