Atm, the push_to_hub script requires the local data folder to be named <data-dir>/<repo-id>_raw. Imo this creates unnecessary friction for the user. I had to create some symlinks before I could create and upload my dataset.
I would suggest to simply use data-dir for the parent directory of the local data folder, and to provide a seperate output dir parameter for local storage.
Atm, the push_to_hub script requires the local data folder to be named
<data-dir>/<repo-id>_raw
. Imo this creates unnecessary friction for the user. I had to create some symlinks before I could create and upload my dataset.I would suggest to simply use
data-dir
for the parent directory of the local data folder, and to provide a seperate output dir parameter for local storage.