cernopendata / cernopendata-client

CERN Open Data command-line client
http://cernopendata-client.readthedocs.io/
GNU General Public License v3.0
10 stars 9 forks source link

download-files: --verify each file as they are downloaded #63

Closed tiborsimko closed 4 years ago

tiborsimko commented 4 years ago

Currently, the files are being verified at the end of a download:

$ cernopendata-client download-files --recid 5500 --filter-range 1-3 --verify
==> Downloading file 1 of 3
  -> File: ./5500/BuildFile.xml
  -> Progress: 0/0 kiB (100%)
==> Downloading file 2 of 3
  -> File: ./5500/HiggsDemoAnalyzer.cc
  -> Progress: 81/81 kiB (100%)
==> Downloading file 3 of 3
  -> File: ./5500/List_indexfile.txt
  -> Progress: 1/1 kiB (100%)
==> Verifying downloaded files for record 5500.
==> Verifying file BuildFile.xml...
  -> expected size 305, found 305
  -> expected checksum adler32:ff63668a, found adler32:ff63668a
==> Verifying file HiggsDemoAnalyzer.cc...
  -> expected size 83761, found 83761
  -> expected checksum adler32:f205f068, found adler32:f205f068
==> Verifying file List_indexfile.txt...
  -> expected size 1669, found 1669
  -> expected checksum adler32:46a907fc, found adler32:46a907fc
==> Success!

It would be good to improve this so that each file is immediately verified after it is downloaded:

This is so that the user can spot troubles early, not only after downloading all the files.

_Originally posted by @tiborsimko in https://github.com/cernopendata/cernopendata-client/pull/58#discussion_r502380191_