search

bigscience-workshop / data_tooling

Tools for managing datasets for governance and training.
Apache License 2.0
77 stars 48 forks source link

add push to hub slurm script #377

Closed SaulLu closed 2 years ago

SaulLu commented 2 years ago

This slurm script sum up a part of the strategy used to push the pseudo crawl dataset to the Hub.

cc @thomasw21 (I can't ask for a review)