search

NDAR / nda-tools

Python package for interacting with NDA web services. Used to validate, submit, and download data to and from NDA.
MIT License
47 stars 22 forks source link

Download shortage #97

Open Wzm111571 opened 3 months ago

Wzm111571 commented 3 months ago

I used nda-tools to download the ABCD data fmri data, but the downloaded single file size is smaller than the public size, and it shows that the download has ended. Is it a network problem or a connection problem with nda?

gregmagdits commented 3 months ago

The size of a package as shown in NDA is likely incorrect because the process falls back to providing an estimate when sizes for all the files in a package are not synced with our s3 repo. This has confused other users in the past and we have an internal ticket to correct the source of the disconnect but until then i advise users that sizes of packages as shown in NDA are estimates only. There is a mode in nda tools to check the sizes of the files on disk with the sizes as reported by AWS S3. You can rerun the downloadcmd with the --verify flag to perform this check.