openai finetune-transformer-lm issues

openai / finetune-transformer-lm

Code and model for the paper "Improving Language Understanding by Generative Pre-Training"

https://s3-us-west-2.amazonaws.com/openai-assets/research-covers/language-unsupervised/language_understanding_paper.pdf

MIT License

2.13k stars 499 forks source link

issues

Newest

Newest Most commented Recently updated Oldest Least commented Least recently updated

The Conv1d over Linear?

#49 maciejbalawejder opened 2 years ago
0
lm_h = tf.reshape(h[:, :-1], [-1, args.n_embd])

#48 lshowway opened 4 years ago
0
what does the variable n_ctx mean ?

#47 lshowway opened 4 years ago
0
great dataset

#46 amirakazadeh opened 5 years ago
0
Why are the "wrong" sentences are learned during training via LM?

#45 fabiang7 opened 5 years ago
0
How to use the current model for classification task?

#44 dhirajpatnaik16297 opened 5 years ago
0
Why do we need to apply mask while fine tuning?

#43 pranoy-k opened 5 years ago
0
Is it possible to edit the code based on this project to train from scratch?

#42 guotong1988 opened 5 years ago
0
Where can I get these two datasets?

#41 ghost closed 5 years ago
4
Expanded README, added requirements.txt

#40 rjurney opened 5 years ago
0
Error!! :(

#39 amirakazadeh opened 5 years ago
10
have you ever try a bigger corpus

#38 ZizhenWang opened 5 years ago
1
Using it as a Language model

#37 mdasadul opened 5 years ago
0
About the non-determinism due to GPU ops

#36 OanaMariaCamburu opened 5 years ago
0
update README with repo status

#35 christopherhesse closed 5 years ago
0
Is the LM pre-trained with a _start_ symbol?

#34 OanaMariaCamburu opened 5 years ago
0
Adds requirements.txt

#33 gabbard opened 5 years ago
0
What is training accuracy of the language model?

#32 peace195 opened 5 years ago
1
Question about "rf" parameter in conv1d() function

#31 XisnZhang opened 5 years ago
0
Add and change implementation for RACE dataset

#30 chengchingwen opened 5 years ago
0
What is the specific formula for learning rate used in adam optimizer during pre-train?

#29 ShuGao0810 opened 5 years ago
0
Do you ever try your model on ROCstory training dataset?

#28 Brandonnogithub opened 5 years ago
0
How to pretrain the model?

#27 ShuGao0810 opened 5 years ago
1
Source Exhausted

#26 Brandonnogithub closed 5 years ago
1
Position embedding matrix Wp was not used in the code?

#25 thanhnguyentang closed 5 years ago
1
help to understand bpe logic

#24 BogdanDidenko closed 5 years ago
1
Universal Transformer as base architecture

#23 rodgzilla opened 5 years ago
0
How to deal with logits from position indices in the output layer?

#22 xiaoda99 opened 6 years ago
1
BUG in utils.py

#21 YichenGong closed 5 years ago
2
how could i train the language with chinese

#20 wykdg opened 6 years ago
3
Using conv1d with kernel size 1

#19 ollmer closed 5 years ago
2
transfer learning code

#18 xuy2 closed 6 years ago
5
0.77 dev accuracy on quora question pairs, but 0.91 is the state of the art

#17 weizier closed 6 years ago
1
Convolution layer filter width

#16 sharpsy closed 5 years ago
1
Cannot reproduce this experiment

#15 Polynomia closed 5 years ago
4
zip argument error

#14 mehdimashayekhi closed 6 years ago
0
Any timeline to release the code to train the LM + finetune on the other 11 tasks?

#13 Franck-Dernoncourt opened 6 years ago
2
What is the meaning of M in the inputs

#12 ioana-blue closed 6 years ago
2
Cannot reproduce RACE score

#11 sugiyama-hiroaki closed 6 years ago
9
implementation of ExponentialMovingAverage is not correct

#10 FrankWork closed 6 years ago
1
Question about the shape of `X_train`

#9 FrankWork opened 6 years ago
1
Ben/add avg to early stopping

#8 benleetownsend closed 6 years ago
0
Unable to adapt language model

#7 chiayewken closed 6 years ago
1
Madison/save load

#6 madisonmay closed 6 years ago
0
Madison/cleanup

#5 madisonmay closed 6 years ago
0
Concatenating context and embeddings?

#4 windweller closed 6 years ago
1
CPU support

#3 throwaway73 opened 6 years ago
6
Supported languages

#2 ErikTromp opened 6 years ago
1
Language Model Generation Example

#1 aykutfirat closed 6 years ago
4