lucidrains linear-attention-transformer issues

lucidrains / linear-attention-transformer

Transformer based on a variant of attention that is linear complexity in respect to sequence length

MIT License

668 stars 64 forks source link

issues

Newest

Newest Most commented Recently updated Oldest Least commented Least recently updated

[Feature request] Self-attention with Persistent Memory

#21 MarcusLoppe closed 2 months ago
1
implemented mup

#20 thomasfortin1 closed 4 months ago
1
Image linear attention reference

#19 pravn opened 6 months ago
0
Why dim != dim_head * heads?

#18 zzczzc20 opened 6 months ago
0
How to perform training?

#17 pangshengwei opened 2 years ago
0
Is the causal attention really works here?

#16 charlesxu90 opened 2 years ago
2
Tooooo many functions added, but no annotations

#15 charlesxu90 opened 2 years ago
0
Scaling factors

#14 radandreicristian closed 2 years ago
1
Challenge in replacing SelfAttention with ImageLinearAttention in Vision Transformer

#13 monajalal opened 2 years ago
0
.

#12 monajalal closed 2 years ago
0
ImageLinearAttention showcase

#11 monajalal closed 2 years ago
0
Causal linear attention from which paper ? please tell me thx

#10 hquzhuguofeng opened 3 years ago
0
dalle

#9 adamonkey closed 3 years ago
1
Questions on the implementation of a linear variant and reference

#8 scaomath closed 3 years ago
1
Where does this constant come from?

#7 aluo-x closed 3 years ago
1
Loss returns Nan

#6 terencenwz opened 3 years ago
3
causal = True

#5 wajihullahbaig opened 3 years ago
2
Autopadder doesn't work with LinearAttentionTransformer

#4 jamarju closed 3 years ago
1
Positional encoding?

#3 matthew-jurewicz closed 4 years ago
2
[Question] Merging with Trans-XL?

#2 gaceladri opened 4 years ago
40
seq2seq decoder ids

#1 ghost closed 4 years ago
4