girder / large_image

Python modules to work with large multiresolution images.
http://girder.github.io/large_image/
Apache License 2.0
190 stars 41 forks source link

Harden marking S3 uploads as large images #1579

Closed manthey closed 1 month ago

manthey commented 1 month ago

The S3 assetstore upload process saves the uploaded file potentially before the upload is finalized in S3. Reading the file immediately can report that the S3 key does not exist. When detecting uploaded files for S3, check if the S3 reported size is the same as the uploaded size. If not, poll and wait before checking.