rtweeks / s3-sysbackup

An AWS S3-backed, Unix-philosophy backup system
Other
1 stars 0 forks source link

Gzipped files upload repeatedly #1

Closed rtweeks closed 3 years ago

rtweeks commented 3 years ago

Using gzip.GzipFile seems to result in varying output content for the same input. Because the only (somewhat) effective mechanism available for evaluating whether a given S3 content object version has the desired content is consulting the ETag -- which is the MD5 of the uploaded (in this case, gzipped) content -- getting varying output from the gzipping process for the same input defeats the time- and space-folding intent.