Open ejm714 opened 2 years ago
Hey, @sambujangfofana and I are students from the University of Michigan. We are currently working on a project wherein we have to contribute to a Github repository(https://eecs481.org/hw6.html). We are pretty interested in this issue and would want to work on it. We hope to submit a pull request this week. Could we be assigned this issue?
Right now, we check that there are at least some filepaths are valid as part of the TrainConfig validation, which happens before we confirm with the user that the config is correct (when using the CLI).
This is not ideal if you're training on a large number of mounted files (e.g. 100,000), as the filepath check can take around an hour in that case. It would be better if we could surface the config before doing any long running checks, so the user can review and adjust any needed things (e.g. add a cache directory).