Add support to reuse transformer weights across long sequences

arnor-sigurdsson / EIR

A toolkit for training deep learning models on genotype, tabular, sequence, image, array and binary data.

https://eir.readthedocs.io/

GNU Affero General Public License v3.0

24 stars 5 forks source link

Closed arnor-sigurdsson closed 2 years ago

arnor-sigurdsson commented 2 years ago

Used by sliding across a longer input sequence, while still encoding global position.