search

Duke-GCB / DukeDSClient

Command line program to allow uploading, downloading, and managing projects in the duke-data-service.
MIT License
5 stars 6 forks source link

Reported size does not match size error when downloading #292

Closed johnbradley closed 4 years ago

johnbradley commented 4 years ago

A user was trying to download a project and received the following error.

Error 400 on /projects/<project-id>/files
Reason:reported size does not match size computed by StorageProvider
Suggestion:You must begin a new upload process

This error seems to be related to only one of the files which somehow has a different size in the backing store than what is expected.

I believe re-uploading this file would fix the problem. Reaching out to the DukeDS team.

johnbradley commented 4 years ago

Details from the DukeDS team the upload in question has the wrong number of chunks. The size of the file does not match the total size of the chunks. Looking at the ddsclient code I do not see how this would happen. When uploading chunks of a file the main thread waits for messages that the appropriate amount has been sent: https://github.com/Duke-GCB/DukeDSClient/blob/3a4c7acc6bbf337590b90de7c23b2cdeaa09e4d2/ddsc/core/util.py#L344-L358

I made a suggestion to the DukeDS team that they could verify an upload before completing it. It is unclear if and when they would implement that check. For now I will can add a check in ddsclient to verify the upload before completing it.

johnbradley commented 4 years ago

Future occurrences of this problem will be prevented by #293.