Open anthonyfok opened 3 years ago
For (my) reference, curl author Daniel Steinberg explains the retry options in depth: https://daniel.haxx.se/blog/2020/03/24/curl-ootw-retry-max-time/
--retry-connrefused
may be a good candidate to add too. It was added in 7.52.0. curl included Ubuntu 18.04 and above is new enough.
Idea: Use git partial clone and sparse-checkout to get just the pointers to *.csv in the original repo to quickly get the SHA256 checksums. Something like the following:
git clone --filter=blob:none --sparse git@github.com:OpenDRR/openquake-inputs.git
cd openquake-inputs
git sparse-checkout add '*.csv'
As a follow up to Issue #66 / PR #82, it would be nice if:
Previous notes (copied from #66):
Check for type:
file
vsdir
--no-act
or--dry-run
or something similar for add_data.sh to "predict" how much files would be downloaded before download.Get sha256 checksum (Git LFS only)
Use a repo for compressed cached copies?