Consistency Improvements

[iglesiasbrandon]

Summary:

We want to improve our consistency model when downloading an object in the presence of concurrent uploads.
Currently, when replacing an object what happens is the object is deleted then the new one is uploaded. This means another observer will see the old object, then nothing, then the new object.

Pain Point:

Object replace is not atomic.

Intended Outcome:

• All object uploads are atomic, specifically during an object replace action.
• An observer should see the old object, then the new; nothing more nothing less.

[iglesiasbrandon]

from the other consistency improvements epic:

After we improve the replace object consistency we need to make sure we can declare success and market strong consistency across our product. #18 should be sufficient, but we need to confirm.

Acceptance criteria: all Uplink operations provide at least strong read-after-write consistency (https://aws.amazon.com/blogs/aws/amazon-s3-update-strong-read-after-write-consistency/), but it would be better to be higher up on https://jepsen.io/consistency.

[iglesiasbrandon]

Milestone: https://github.com/storj/storj/milestone/14

[iglesiasbrandon]

final ticket needed: https://github.com/storj/storj-private/issues/103