issues
search
openai
/
finetune-transformer-lm
Code and model for the paper "Improving Language Understanding by Generative Pre-Training"
https://s3-us-west-2.amazonaws.com/openai-assets/research-covers/language-unsupervised/language_understanding_paper.pdf
MIT License
2.13k
stars
499
forks
source link
issues
Newest
Newest
Most commented
Recently updated
Oldest
Least commented
Least recently updated
The Conv1d over Linear?
#49
maciejbalawejder
opened
2 years ago
0
lm_h = tf.reshape(h[:, :-1], [-1, args.n_embd])
#48
lshowway
opened
4 years ago
0
what does the variable n_ctx mean ?
#47
lshowway
opened
4 years ago
0
great dataset
#46
amirakazadeh
opened
5 years ago
0
Why are the "wrong" sentences are learned during training via LM?
#45
fabiang7
opened
5 years ago
0
How to use the current model for classification task?
#44
dhirajpatnaik16297
opened
5 years ago
0
Why do we need to apply mask while fine tuning?
#43
pranoy-k
opened
5 years ago
0
Is it possible to edit the code based on this project to train from scratch?
#42
guotong1988
opened
5 years ago
0
Where can I get these two datasets?
#41
ghost
closed
5 years ago
4
Expanded README, added requirements.txt
#40
rjurney
opened
5 years ago
0
Error!! :(
#39
amirakazadeh
opened
5 years ago
10
have you ever try a bigger corpus
#38
ZizhenWang
opened
5 years ago
1
Using it as a Language model
#37
mdasadul
opened
5 years ago
0
About the non-determinism due to GPU ops
#36
OanaMariaCamburu
opened
5 years ago
0
update README with repo status
#35
christopherhesse
closed
5 years ago
0
Is the LM pre-trained with a _start_ symbol?
#34
OanaMariaCamburu
opened
5 years ago
0
Adds requirements.txt
#33
gabbard
opened
5 years ago
0
What is training accuracy of the language model?
#32
peace195
opened
5 years ago
1
Question about "rf" parameter in conv1d() function
#31
XisnZhang
opened
5 years ago
0
Add and change implementation for RACE dataset
#30
chengchingwen
opened
5 years ago
0
What is the specific formula for learning rate used in adam optimizer during pre-train?
#29
ShuGao0810
opened
5 years ago
0
Do you ever try your model on ROCstory training dataset?
#28
Brandonnogithub
opened
5 years ago
0
How to pretrain the model?
#27
ShuGao0810
opened
5 years ago
1
Source Exhausted
#26
Brandonnogithub
closed
5 years ago
1
Position embedding matrix Wp was not used in the code?
#25
thanhnguyentang
closed
5 years ago
1
help to understand bpe logic
#24
BogdanDidenko
closed
5 years ago
1
Universal Transformer as base architecture
#23
rodgzilla
opened
5 years ago
0
How to deal with logits from position indices in the output layer?
#22
xiaoda99
opened
6 years ago
1
BUG in utils.py
#21
YichenGong
closed
5 years ago
2
how could i train the language with chinese
#20
wykdg
opened
6 years ago
3
Using conv1d with kernel size 1
#19
ollmer
closed
5 years ago
2
transfer learning code
#18
xuy2
closed
6 years ago
5
0.77 dev accuracy on quora question pairs, but 0.91 is the state of the art
#17
weizier
closed
6 years ago
1
Convolution layer filter width
#16
sharpsy
closed
5 years ago
1
Cannot reproduce this experiment
#15
Polynomia
closed
5 years ago
4
zip argument error
#14
mehdimashayekhi
closed
6 years ago
0
Any timeline to release the code to train the LM + finetune on the other 11 tasks?
#13
Franck-Dernoncourt
opened
6 years ago
2
What is the meaning of M in the inputs
#12
ioana-blue
closed
6 years ago
2
Cannot reproduce RACE score
#11
sugiyama-hiroaki
closed
6 years ago
9
implementation of ExponentialMovingAverage is not correct
#10
FrankWork
closed
6 years ago
1
Question about the shape of `X_train`
#9
FrankWork
opened
6 years ago
1
Ben/add avg to early stopping
#8
benleetownsend
closed
6 years ago
0
Unable to adapt language model
#7
chiayewken
closed
6 years ago
1
Madison/save load
#6
madisonmay
closed
6 years ago
0
Madison/cleanup
#5
madisonmay
closed
6 years ago
0
Concatenating context and embeddings?
#4
windweller
closed
6 years ago
1
CPU support
#3
throwaway73
opened
6 years ago
6
Supported languages
#2
ErikTromp
opened
6 years ago
1
Language Model Generation Example
#1
aykutfirat
closed
6 years ago
4