search

mgrankin / ru_transformers

Apache License 2.0
776 stars 108 forks source link

Are you the first who adopts GPT-2 to Russian? #1

Closed utahman closed 4 years ago

mgrankin commented 4 years ago

I’m sure I’m not. @rkfg have tried to to that a long ago and composed a repo https://github.com/rkfg/gpt-2 I believe many people did that in private companies.

But, I haven’t seen any publicly released model weights. And I haven’t seen anyone to use progressive layer unfreezing - this is the correct way to use transfer learning. And nobody have tried to train PyTorch/Transformers model on TPU. So this repo is adding some value.

stale[bot] commented 4 years ago

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.