Detect S3 network transmission errors

[GoogleCodeExporter]

S3 returns an MD5 hash of what it received (PUT case) or what it
transmitted (GET) case. s3fs should check this response header (ETag) or
supply a Content-MD5 request header to guard against the possibility of
data corruption.

There is at least one documented, Amazon-acknowledged incident of S3 data
corruption due to network transmission errors (load-balancing hardware
failure in this case):

http://developer.amazonwebservices.com/connect/message.jspa?messageID=93408#9340
8

Original issue reported on code.google.com by marcin_...@yahoo.com on 24 Jun 2008 at 9:11

[GoogleCodeExporter]

This issue might be tied into issue #23 ?

Original comment by dmoore4...@gmail.com on 9 Mar 2011 at 7:55

[GoogleCodeExporter]

This is my only worry regarding issue #23, size and mtime don't technically 
mean the compared files are _exactly_ the same. I've got a patch just about 
ready for issue #23. Personally, I think size and mtime are good enough, though 
some might argue otherwise.

Original comment by ben.lema...@gmail.com on 9 Mar 2011 at 9:13

[GoogleCodeExporter]

This issue is fixed by v1.69.

Added new option enable_content_md5, you can specify this option for checking 
uploaded object.

I closed this issue.
If you find another bug, please post new issue.

Thanks,

Original comment by ggta...@gmail.com on 16 May 2013 at 7:52

• Changed state: Fixed