How to train/fine tune the model with multiple GPUs?

graykode / gpt-2-Pytorch

Simple Text-Generator with OpenAI gpt-2 Pytorch Implementation

MIT License

963 stars 225 forks source link

How to train/fine tune the model with multiple GPUs? #21

Open gudiandian opened 4 years ago

gudiandian commented 4 years ago

I have pulled the code from branch train. Is there a way to train or fine tune the GPT-2 model with data parallelism on multiple GPUs? Thanks for your help.