storage: calculate block size for multipart upload

inveniosoftware / invenio-s3

S3 file storage support for Invenio.

https://invenio-s3.readthedocs.io

MIT License

0 stars 16 forks source link

Closed egabancho closed 4 years ago

egabancho commented 4 years ago

When the file is too big, the block size doesn't get updated and we reach multipart maximum number of parts raising the corresponding exception. Calculating the block size for each operation based on the actual file size seems to solve the issue. (closes #11)

egabancho commented 4 years ago

@ppanero this has been tested in a "real world" situation already. If you, or anyone else, can give it an eyeball test, it should be read to fly.

cc @Herrner