search

yevhen / Streamstone

Event store for Azure Table Storage
Other
395 stars 64 forks source link

Idempotency Edge Conditions #11

Closed gregoryyoung closed 9 years ago

gregoryyoung commented 9 years ago

I am not finding any tests on some of the bizarre edge conditions that happen around idempotency. As an example:

I write in a batch events 1,3,7 I then write a batch 2,7,14

Is this really an idempotent write? or is it an error.

yevhen commented 9 years ago

Hi man! I'm really proud that you've starred my project. Cool to see you here :)

Yes, you're absolutely right. I should probably reword description. I do provide duplicate event detection feature which could be used by users to implement idempotence but writes are not automatically idempotent.

The user will get back DuplicateEventException if the event with the same id was previously stored. The offending event will be passed as the payload of exception. The user could then remove it from batch and retry (or just cancel).

I made the choice to implement it this way due to the following reasons:

  1. There is no any other way to implement this feature on WATS than using retries.
  2. WATS .NET SDK reports only a first conflict in a batch. So the worst case if using retries is, I might end up doing a retry per every event in a batch. Which sucks.
  3. I hate automatic retries done behind my back :)

I think it's important for end-user to be aware of duplicates. The high rate of duplicates could mean a severe problem somewhere else, and that better be checked.

P.S. I don't even have the test for 2,7,2. I should probably add it :smile:

yevhen commented 9 years ago

Forgot to mention. That behavior is illustrated here https://github.com/yevhen/Streamstone/blob/master/Source/Example/Scenarios/S09_Handling_duplicates.cs

gregoryyoung commented 9 years ago

This code just feels so weird for me in terms of calling it idempotency can I make it more fun and throw an IdempotencyException? :)

On Wed, Jun 3, 2015 at 5:47 PM, Yevhen Bobrov notifications@github.com wrote:

Forgot to mention. That behavior is illustrated here https://github.com/yevhen/Streamstone/blob/master/Source/Example/Scenarios/S09_Handling_duplicates.cs

— Reply to this email directly or view it on GitHub https://github.com/yevhen/Streamstone/issues/11#issuecomment-108465690.

Studying for the Turing test

yevhen commented 9 years ago

Well, the code looks absolutely legitimate but the message is not :smile:

No worries, I'll change it appropriately to not confuse the people.

yevhen commented 9 years ago

@gregoryyoung thanks, your feedback is incredibly important for me! Would be happy to see more issues spotted by 'the godfather' :+1:

P.S. This project is really young :blush:

yevhen commented 9 years ago

@gregoryyoung does it make more sense now? May I close this issue? :smile:

gregoryyoung commented 9 years ago

Yep go for it. I think its just an expectation mismatch :)

On Wed, Jun 3, 2015 at 7:41 PM, Yevhen Bobrov notifications@github.com wrote:

@gregoryyoung https://github.com/gregoryyoung does it make more sense now? May I close this issue? [image: :smile:]

— Reply to this email directly or view it on GitHub https://github.com/yevhen/Streamstone/issues/11#issuecomment-108517602.

Studying for the Turing test

yevhen commented 9 years ago

Cool. Thanks, man!