Closed willkg closed 10 months ago
Currently, fetch-data downloads data in serial. It'd be much faster if it could download in parallel.
Maybe something like?:
cat crashids.txt | fetch-data --workers=5 ...
Where workers defaults to 1.
workers
We'd need to change the output so it's possible to follow the work easier. Maybe we require an auth token to use this option, too.
Currently, fetch-data downloads data in serial. It'd be much faster if it could download in parallel.
Maybe something like?:
Where
workersdefaults to 1.We'd need to change the output so it's possible to follow the work easier. Maybe we require an auth token to use this option, too.