search

minimaxir / aitextgen

A robust Python tool for text-based AI training and generation using GPT-2.
https://docs.aitextgen.io
MIT License
1.84k stars 220 forks source link

Gpt-J #223

Open robitec97 opened 1 year ago

robitec97 commented 1 year ago

Can i train gpt-j from scratch?

Vectorrent commented 1 year ago

Yes. It's not going to work out-of-the-box with AITextGen, but you should be able to make it work, with a little bit of tweaking. GPT-J is a part of the Huggingface Transformers module, so training should be nearly identical to how you'd train GPT-2/GPT-Neo with AITextGen.