Training Regime and Backprop

[friesel]

Love your project and hope very much it's not impolite to ask whether you intend to publish the learning regime as well so that I might try my luck with my own text data (training on the text corpus and the later the classifier on top). Even "dirty" code would do, just to get a starting point.

Would be very happy to hear from you

[zhaoyingjun]

i want train it in chinese

[jonny-d]

Code to train this model 'from scratch' using data-parallelism across multiple GPUs: https://github.com/jonnykira/openai_reproduction