tsileo / bakthat

Bakthat is a MIT licensed backup framework written in Python, it's both a command line tool and a Python module that helps you manage backups on Amazon S3/Glacier and OpenStack Swift. It automatically compress, encrypt (symmetric encryption) and upload your files.
http://bakthat.readthedocs.org/en/latest/
MIT License
491 stars 67 forks source link

Use boto multipart upload for uploading large files to S3 #84

Closed shanx closed 4 months ago

shanx commented 7 years ago

This patch allows uploading large files (5gb or larger) to Amazon S3 using boto's multipart transfer. Effectively mitigating #75 and boto/boto#2207. When a file is larger then 2gb then the file will be uploaded in chunks of 100mb. Files smaller then 2gb will be uploaded using the already implemented methodology