search

elastic / elasticsearch-cloud-aws

AWS Cloud Plugin for Elasticsearch
https://github.com/elastic/elasticsearch/tree/master/plugins/discovery-ec2
577 stars 181 forks source link

Check MD5 while doing snapshot #188

Closed dadoonet closed 9 years ago

dadoonet commented 9 years ago

There is a feature available in S3 that clients can use to ensure data integrity on upload. Whenever an object is PUT to an S3 bucket, the client is able to get back the MD5 base64 encoded and check that it's the same MD5 as the local one.

For reference, please see the S3 PutObject API.

Closes #186.

rmuir commented 9 years ago

This is a great feature! Although we have our own internal checksums, by using their MD5 we know immediately at snapshot time that it got corrupted in transit somehow, rather than the user finding out about it much much later when they try to restore it (possibly because of some other disaster).

I left one comment, more on the style side, but it looks great to me.