search

LoreDema / GePpeTto

GePpeTto, is a generative language model for Italian, built using the GPT-2 architecture.
Other
13 stars 1 forks source link

train from scratch #1

Open g-i-o-r-g-i-o opened 2 years ago

g-i-o-r-g-i-o commented 2 years ago

Is there any resource that explains how to train from scratch? I'm interested on building geppetto on a smaller dataset. I suppose that italian requires a lot of work with the tokenizers??

LoreDema commented 2 years ago

Hi @GianniGi check this out: https://huggingface.co/blog/how-to-train

I think it pretty much covers it up.