Put Object (if changed)

[trajche]

For Buckets that have enabled Versioning, it would be useful to have a "Upsert Object" (insert or update).

Currently, if you do a Put Object it will add a new Version of the object no matter if it's the same contents. The goal would be to check the contents and compare.

There are two ways that comparison could be done:

• MD5 calculation of the file contents (and compare with current version)
• Use the ETAG param for faster checking (this is available in headers when getting S3 object) s3-etag library

The S3 ETAG is also a MD5 calculation, but only for non-multipart files. Not sure how it works for multipart uploads.

[rristov60]

This issue is closed with #13 . Upon item insertion, first MD5 hash is calculated for the body of the item, and it is compared to the current ETag for the object if the object already exists. If they don't match, it means that the contents of the file are changed and the object is put. If versioning is enabled, and the object exists, new version of it will be created. This is both for Put Object and Put Objects. Please note that this is valid only for objects that have MD5 ETags. If the ETags are not MD5, each time the object will be inserted. More details about ETags can be found here.

[trajche]

Re-opening this one, as it seems to overwrite files after the 1.3.0 release.

[trajche]

Btw, in order to allow for speed, perhaps there can be a property/checkbox (Upsert - Only Put Object if the MD5 is different), which if 'true' then will process Upserts, otherwise it will be Put.

This will allow for users that don't want Upserting to save on a HEAD Object call for every object.

[trajche]

Closing this until futher notice. Seems there is some issue with md5 / etag matching. Will debug more and re-open if I can reproduce.