Closed k8tems closed 7 months ago
I think I got it.
I just had to change the webdataset_path
config from a list to a string.
from:
webdataset_path:
- file:/tmp/mbl_2024_02_14_13_12.tar
to:
webdataset_path: file:/tmp/mbl_2024_02_14_13_12.tar
According to the readme in train, the config supports local files.
However, when I run the training script, I get the following error and it seems like the script is stuck in an infinite loop trying to copy from aws.
output(config included):
The logs go on forever with most of it being, "Unknown options: -"