opencadc / cadctools

Client tools for using CADC services
3 stars 11 forks source link

Support for resumable downloads (Cadc 10362) #183

Closed andamian closed 2 years ago

andamian commented 2 years ago

This features supports resumable file downloads with cadcget. The feature is implemented in cadcutils so it could probably be used with vos or other applications.

The feature only works with services that support Range requests ((RFC7233). It also requires Content-Length and Digest HTTP headers to asses whether the file has been correctly transferred.

How it works: File is downloaded in the destination directory under a temporary file name of the form (`cadcget--). If such a file already exists at the destination, a range request is send to the server instead and the download is resumed from where it left. Upon successful download, the temporary file is moved to the actual destination file name.

Notes:

Fix for #184

codecov[bot] commented 2 years ago

Codecov Report

Merging #183 (79c5c71) into master (b901f44) will increase coverage by 0.32%. The diff coverage is 93.00%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master     #183      +/-   ##
==========================================
+ Coverage   87.28%   87.60%   +0.32%     
==========================================
  Files          31       31              
  Lines        3515     3542      +27     
==========================================
+ Hits         3068     3103      +35     
+ Misses        447      439       -8     
Impacted Files Coverage Δ
cadcutils/cadcutils/old_get_cert.py 70.76% <0.00%> (ø)
cadcutils/cadcutils/net/ws.py 91.14% <93.75%> (+0.68%) :arrow_up:
cadcdata/cadcdata/storageinv.py 83.13% <100.00%> (+0.53%) :arrow_up:
cadcutils/cadcutils/net/auth.py 87.59% <100.00%> (+0.09%) :arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more Δ = absolute <relative> (impact), ø = not affected, ? = missing data Powered by Codecov. Last update b901f44...79c5c71. Read the comment docs.