search

LoicGrobol / zeldarose

Train transformer-based models.
https://zeldarose.readthedocs.io
Other
28 stars 3 forks source link

Switch to 🤗 datasets #7

Closed LoicGrobol closed 3 years ago

LoicGrobol commented 3 years ago

Switching to 🤗 datasets to avoid loading the trainset in RAM for every ddp process. A lot of changes along the way