issues
search
dingo-actual
/
infini-transformer
PyTorch implementation of Infini-Transformer from "Leave No Context Behind: Efficient Infinite Context Transformers with Infini-attention" (https://arxiv.org/abs/2404.07143)
MIT License
271
stars
22
forks
source link
issues
Newest
Newest
Most commented
Recently updated
Oldest
Least commented
Least recently updated
Memory issues with core implementation
#20
dingo-actual
opened
3 months ago
0
Update docs and example to include positional embeddings
#19
dingo-actual
closed
4 months ago
1
Added docstrings and comments to modinfiniformer example
#18
willtai
closed
4 months ago
0
Added UpdateType, docstrings, and comments
#17
willtai
closed
4 months ago
2
Updated code files
#15
muditbhargava66
closed
4 months ago
1
Add comments to the example script
#14
dingo-actual
closed
4 months ago
4
Add comments, docstrings and citations for activations
#13
dingo-actual
closed
4 months ago
1
Memory savings?
#12
jlamprou
closed
4 months ago
2
Add multiple options for nonlinear activation
#11
rtaylor-rx-m
closed
4 months ago
1
Handle batch input inference for MoD Infini-Former more gracefully
#10
dingo-actual
opened
4 months ago
3
Add positional embeddings
#9
dingo-actual
closed
4 months ago
5
Change MoD sampled sequence length logic to allow for (almost) arbitrary `sampling_factor`
#8
dingo-actual
closed
4 months ago
1
Update readme to include MoD Infini-Former instructions
#7
dingo-actual
closed
4 months ago
3
Incorporate Mixture-of-Depths
#6
dingo-actual
closed
4 months ago
1
q, k, v projection inside the loop
#5
usryokousha
closed
4 months ago
2
Causal Masking.
#4
vmarinowski
closed
4 months ago
4
make package installable
#3
Borda
closed
4 months ago
0
inference issue
#2
winglian
closed
4 months ago
12
fix sigma_k usage in update and delay it
#1
amitportnoy
closed
4 months ago
0