12 layer (huggingface gpt-2 equivalent) ctrl model?

salesforce / ctrl

Conditional Transformer Language Model for Controllable Generation

https://arxiv.org/abs/1909.05858

BSD 3-Clause "New" or "Revised" License

1.87k stars 208 forks source link

12 layer (huggingface gpt-2 equivalent) ctrl model? #82

Open zichaow opened 3 years ago

zichaow commented 3 years ago

Hi ! Wondering if smaller-sized, pre-trained ctrl models, e.g., the same size as the gpt-2 model on huggingface, are available - ideal for smaller-scale experiments without large-memory GPUs :)) Thanks !