tatp22 linformer-pytorch issues

tatp22 / linformer-pytorch

My take on a practical implementation of Linformer for Pytorch.

https://arxiv.org/pdf/2006.04768.pdf

MIT License

400 stars 36 forks source link

issues

Newest

Newest Most commented Recently updated Oldest Least commented Least recently updated

Question about how to modify to predict on a series of sparse number

#27 JonasLi-19 opened 6 months ago
0
Question: Is Linformer permutation equivariant (set-operation)?

#26 nmakes opened 1 year ago
5
Loss goes to 0 when using LinformerLM

#25 terencenwz closed 3 years ago
2
Different number of tokens and Character Level Modeling

#24 wajihullahbaig closed 3 years ago
2
Error with DistributedDataParallel and parameter_sharing="layerwise"

#23 blizda closed 3 years ago
2
Error with DistributedDataParallel

#22 blizda closed 3 years ago
2
See issue #20

#21 tatp22 closed 3 years ago
0
Error when using method="no_params" and GPU, because E and F incorrectly remain on CPU

#20 RaivoKoot closed 3 years ago
4
Use -inf as mask value for the causal mask

#19 kklemon closed 3 years ago
2
Got rid of unneccessary activation

#18 tatp22 closed 3 years ago
0
Added some more dropout

#17 tatp22 closed 3 years ago
0
causal_mask of the decoder

#16 burcehan closed 3 years ago
4
How to interpret the visualization results?

#15 mertyyanik closed 3 years ago
2
embeddings_mask datatype

#14 tongcu closed 3 years ago
1
Any result on any benchmark?

#13 twangnh opened 3 years ago
5
Added masking

#12 tatp22 closed 3 years ago
0
padding mask and attention mask

#11 zackchen-lb closed 3 years ago
10
Enquiry about your implementation

#10 riven314 closed 3 years ago
2
Fixed a possible bug with the data

#9 tatp22 closed 4 years ago
0
Possible bug

#8 tatp22 closed 4 years ago
2
Changed things as mentioned in issue 6

#7 tatp22 closed 4 years ago
0
Composed linear layers?

#6 apeguero1 closed 4 years ago
5
Any performance test on different checkpoint level ?

#5 phongnhhn92 closed 4 years ago
2
Would you like to release the pretrain tutorial?

#4 RyanHuangNLP closed 3 years ago
11
Huggingface

#3 flozi00 closed 4 years ago
3
input seg length

#2 xinqipony closed 4 years ago
1
Will any pretrained linformer models be open sourced?

#1 pchankh closed 4 years ago
1