search

dotnetcore / CAP

Distributed transaction solution in micro-service base on eventually consistency, also an eventbus with Outbox pattern
http://cap.dotnetcore.xyz
MIT License
6.69k stars 1.29k forks source link

Improve consumer message sequence and idempotence #29

Closed yang-xiaodong closed 6 years ago

yang-xiaodong commented 7 years ago

In subscribe method, CAP needs to add these two features.

ah-its-andy commented 7 years ago

Can you describe simply the solution you are working with to improve idempotence?

yang-xiaodong commented 7 years ago

@standardcore In version 2.0, I have wapped the message content with Message class, Message object provider a Id property used to be guarantee message sequence, in consumer method we can check the Id whether it has been processed or not.

yang-xiaodong commented 6 years ago

This feature will be given to the user to ensure.

yang-xiaodong commented 5 years ago

CAP does not implement idempotent for several reasons: The following are the receiving end

There are many reasons why the Consumer method fails. I don't know if the specific scene is blindly retrying or not retrying is an incorrect choice. For example: If the consumer is the debit service, if the execution of the deduction is successful, but the failure to write the debit log, the CAP will judge the consumer execution failure, and try again. If the client does not guarantee idempotency, the framework will retry it, which will inevitably lead to serious consequences for multiple deductions.

The scenario here is also possible. If the Consumer has been successfully executed at the beginning, but for some reason, such as the Broker recovery, and received the same message, the CAP will consider this a new after receiving the Broker message. The message will be executed again by the Consumer. Because it is a new message, the CAP cannot be idempotent at this time.

Since the table of the CAP message is deleted after 1 hour for the successfully consumed message, if the historical message cannot be idempotent. Historically, it means that if the broker has maintained or manually processed some messages for some reason.

Many event-driven frameworks require users to ensure idempotent operations, such as ENode, RocketMQ, etc...

From an implementation point of view, CAP can do some less stringent idempotence, but strict idempotent cannot.