search

huggingface / cosmopedia

Apache License 2.0
458 stars 45 forks source link

Training code for Cosmo-1B? #10

Open jwkirchenbauer opened 9 months ago

jwkirchenbauer commented 9 months ago

Awesome work 🙂

Is there any plan to release the training code for cosmo-1b? Or at least details about what existing repos and framework tools were used?

loubnabnl commented 9 months ago

Hi, we used an internal wrapper around nanotron framework, this is the training config (you will need to adapt it to work with nanotron)