search

openai / finetune-transformer-lm

Code and model for the paper "Improving Language Understanding by Generative Pre-Training"
https://s3-us-west-2.amazonaws.com/openai-assets/research-covers/language-unsupervised/language_understanding_paper.pdf
MIT License
2.15k stars 503 forks source link

What is training accuracy of the language model? #32

Open peace195 opened 6 years ago

peace195 commented 6 years ago

I trained this architecture on Japanese and got only 50% training accuracy. Is it normal here?

I suspect it is not 100% because we predict for entire sequence in this model, not only for n-gram as Word2Vec.

Thank you very much.

lismalaj commented 11 months ago

I trained this architecture on Japanese and got only 50% training accuracy. Is it normal here?

I suspect it is not 100% because we predict for entire sequence in this model, not only for n-gram as Word2Vec.

Thank you very much.

I am a beginner and was wondering how did you get the program running? What was your steps to getting it to work? thank you