guolinke TUPE issues - Githubissues

guolinke / TUPE

Transformer with Untied Positional Encoding (TUPE). Code of paper "Rethinking Positional Encoding in Language Pre-training". Improve existing models like BERT.

MIT License

249 stars 26 forks source link

issues

Newest

Newest Most commented Recently updated Oldest Least commented Least recently updated

Definition of two learnable parameters

#22 tom68-ll opened 1 year ago
4
我终于完成数据预处理了，谢谢您的回答和帮助，我想问一下按照你的参数在4块16g的v100上要跑多久能跑完？

#21 tom68-ll opened 2 years ago
4
Improve pos_embed calculation

#20 ZhiyuanChen opened 2 years ago
0
How to set up input for sentence classification task?

#19 huberemanuel closed 2 years ago
2
No such file or directory: './data-bin/pt-data-0508/dict.txt'

#18 huberemanuel opened 2 years ago
4
question of providing bert model integrated with TUPE

#17 EchoFei333 opened 3 years ago
1
The first column of `dict.txt` is equivalent to the vocabulary learned from fastBPE?

#16 huberemanuel closed 3 years ago
2
How to calculate correlation in Figure 2?

#15 Redaimao opened 3 years ago
1
Fix bug of warnings not imported in mha

#14 ZhiyuanChen closed 3 years ago
0
Fix bug of pad not defined in multihead_attention

#13 ZhiyuanChen closed 3 years ago
0
Discrepancy between the paper and the implementation?

#12 tonyswoo closed 3 years ago
2
Reproduce BERT and TUPE

#11 wyu97 closed 3 years ago
3
About the pre-trained checkpoints

#10 VerdureChen opened 3 years ago
1
some issues

#9 sanwei111 opened 3 years ago
1
Do these codes also fit for encoder-decoder transformer?

#8 SefaZeng opened 3 years ago
3
有没验证过，是pretrain阶段的增益，还是本身这个position-encoding就有增益（不pretrain也有）？

#7 guotong1988 closed 3 years ago
4
Could you please point out the CORE code for TUPE for study? As there are too many fairseq code.

#6 guotong1988 closed 3 years ago
1
A TensorFlow Implementation？

#5 guotong1988 closed 3 years ago
1
About the reset function in attention bias

#4 Erutan-pku closed 3 years ago
4
What's the format of the raw data?

#3 Howal opened 4 years ago
4
我跑你的代码，数据集那些怎么弄，怎么装载字典，我没有dict.txt

#2 wymxz opened 4 years ago
12
相对位置具体配置

#1 PROoshio closed 4 years ago
1