search

wallix / awless

A Mighty CLI for AWS
http://awless.io/
Apache License 2.0
4.97k stars 263 forks source link

Support the Multi-part-upload API in create s3object to allow the upload of files >5GB #223

Open rhmoult opened 6 years ago

rhmoult commented 6 years ago

Hello! I'm excited to try awless, but one of my early use cases is failing. I tried to upload a 6.6 GB file to an S3 bucket in us-east-1 and got the following message: KO create s3object EntityTooLarge: Your proposed upload exceeds the maximum allowed size

Grepping through your code didn't reveal this message in your git repo. Any idea what sets the maximum allowed size? I ask because I didn't run into similar issues with aws-cli with similar sized files in the past.

fxaguessy commented 6 years ago

Hi, Thanks for your feedback. This is not an error throw by awless but by AWS servers. Indeed, they do not support uploads larger than 5GB in a single upload (what we are currently doing in awless). When you have a file from 5GB to 5TB, you need to use the multi-part-upload API (cf https://docs.aws.amazon.com/AmazonS3/latest/dev/UploadingObjects.html)

Thus as this is not yet implemented in awless, for now you can not upload files bigger than 5GB.