what about adding the recent model OpenAI GPT-2 ?

graykode / nlp-tutorial

Natural Language Processing Tutorial for Deep Learning Researchers

https://www.reddit.com/r/MachineLearning/comments/amfinl/project_nlptutoral_repository_who_is_studying/

MIT License

14.31k stars 3.95k forks source link

what about adding the recent model OpenAI GPT-2 ? #15

Closed lianggaoquan closed 5 years ago

lianggaoquan commented 5 years ago

Since the recent model OpenAI GPT-2 may be better than BERT and it would be an important breakthrough in NLP, why not adding its' intuitive and simplest code in this repo ?

graykode commented 5 years ago

@lianggaoquan
Good Issue! Hello! I try to implement here : https://github.com/graykode/gpt-2-Pytorch But, It is hard to convert tensorflow checkpoint file to pytorch gpt-2 concept is not much different from the openAi-gpt. Also, if you run gpt-2 model, https://github.com/huggingface/pytorch-pretrained-BERT this project also good Thanks :D