[Extension Proposal] Auto-compensating saga extension

[xian75]

Description

An extension to develop SAGA/CQRS orchestrators and participants in an easy way. It allows you to focus on the logic of your business instead of think about the way to implement the SAGA pattern. It avoids the effort to write compensation operations because it does by itself automatically. All the data are stored in SQL databases, taking advantage of database constraints validations management. The framework depends on the following Quarkus extensions:

• quarkus-scheduler
• quarkus-rest-jackson
• quarkus-rest-client-jackson
• smallrye-mutiny-vertx-sql-client

Repository name

quarkus-sagacqrs

Short description

An extension for SAGA/CQRS microservices development with SQL databases and automatic compensation operations.

Repository Homepage URL

https://quarkiverse.github.io/quarkiverse-docs/quarkus-sagacqrs/dev/

Repository Topics

• quarkus-extension
• microservices
• saga
• cqrs

Team Members

• xian75

Additional context

No response

[xian75]

I'm interested in leading the development of this extension. A beta version of this extension is already done.

[gastaldi]

@xian75 hey! That looks interesting. Do you have the beta version available somewhere so we can look?

[xian75]

Hey @gastaldi, I added you as collaborator to my private projects. The sagacqrs is the library. Instead, orchestrator and three participants are samples to show how to use the library. Let me know what do you think about it.

[gastaldi]

Adding the links here, so I don't miss them:

• https://github.com/xian75/sagacqrs/
• https://github.com/xian75/orchestrator-ABC/
• https://github.com/xian75/participant-A/
• https://github.com/xian75/participant-B/
• https://github.com/xian75/participant-C/

Any chance to make them public so others can chime in?

[xian75]

Now all repos are public. For any question or an overview about the lib don't hesitate to email me. I'll try to respond ASAP.

[zhfeng]

I think there is the quarkus-narayan-lra and just wonder this proposal extension has any difference since they both implement the SAGA pattern. Thanks!

[xian75]

I didn't know the narayan extension but reading its overview I can say it's very different from my proposal one. Basically, the main difference is that my proposal doesn't require any compensation to be implemented cause the extension do it automatically on your behalf. Unfortunately these days I'm very busy but next days I'll give me a canche to write a deeper overview about my proposal and how it works.

[xian75]

@zhfeng finally I found a bit of time to type something...

Overview The idea behind this library is a 2-steps transaction concept (not a 2PC). During the first step of a saga workflow, the orchestrator "open" a new distributed pending saga transaction asking participants to perform writing (eventually also reading) database operations. According to the saga pattern, each participant executes its duties in an ACID local database transaction. But all those create, update and delete operations don't definitely change the table rows. They just lock and change the "status" of those rows preventing they can be changed by other saga requests. Once all participants have finished their operations, the orchestrator collect their outcomes and change the distributed saga transaction status from pending to commit or rollback. Commit if all participants operations went through, rollback otherwise. A transaction is rolledback even when a timeout of one or more participant occur. This is the end of the first step. The orchestrator is provided by a couple of scheduled jobs. One of this collects all committed and rolledback distributed saga transactions asking participants to "finalize" them: this is the second step. Each participant finalize its database appling the changes to the table rows and unlocking them. Again, the finalization is applied by an ACID local database transaction and all operations are idempotents. Once all participants have finished their operations, the orchestrator collects their outcomes and delete the distributed saga transactions if all participants finish successfully. Otherwise, the orchestrator leaves the distributed saga transactions as they are waiting for the next scheduled job execution. Becuase every distributed saga transaction is indipendent one another, than each scheduled job commonly collects some distributed saga transactions to delete and others to leave. That means it's not necessary that all distributed saga transactions in a single scheduled job must go through to be deleted.

More details The orchestrator creates, updates and deletes distributed saga transactions in its own database, consisting of just one table: "transactions". This table has the following columns:

• uuid: the unique distributed saga transaction identifier
• operation: the name of the operation to be performed by the participants
• outcome: the transaction outcome corresponding to the transaction status
• timeout: the maximum transaction duration in ms
• expire: the expiry datetime (transaction creation datetime + timeout) Instead, the participants manage their entity according to their own database schemas and some "constraints" required by the library. In fact, the library needs that every table must have the following columns in order to perform the 2-steps transaction:
• id: the row id as usual but with no primary key constraint defined, just an index
• optlock: the version number of the row for optimistic lock and concurrency management
• event_uuid: the unique distributed saga transaction identifier created by the orchestrator
• event_state: the status of the row whose value can be DELETED(-1), INSERTING(0), LOGICAL_DELETING(1), DELETING(2), UPDATE_ARCHIVING(3), UPDATING(4), UPDATE_TO_APPLY(5) The no primary key id and consequently missing foreign key constraints can look like too strong. But thinking about the saga pattern, it was born just to break the monolithic database and application into a set of little applications with their own small databases. So, saga pattern naturally breaks the entity relationships within a common monolithic database. About the event_state, all the possible values but DELETED(-1) are transients and live just during the first step of a saga workflow. In fact, after the second "finalization" step, every row has an event_state equal to:
• DELETED(-1): when the row is logically deleted
• null: when the row is finalized and ready to be locked by another distributed saga transaction Basically:
• when the participant needs to create a new row, it creates that one settings its event_state to INSERTING(0)
• when the participant needs to update a row, it marks the old one with an UPDATING(4) event_state and create a new one with the same id and an event_state equals to UPDATE_TO_APPLY(5)
• when the participant needs to delete a row, it marks that one with a DELETING(2) event_state and similarly:
• when the participant needs to update a row archiving the old values, it marks the old row with an UPDATE_ARCHIVING(3) event_state and create a new one with the same id, new values and an event_state equals to UPDATE_TO_APPLY(5)
• when the participant needs to logically delete a row, it marks that one with a LOGICAL_DELETING(1) event_state Toghether with the event_state, the participant marks the rows with the uuid provided by the orchestrator. So that, when the orchestrator asks participants to finalize a certain uuid distributed saga transaction, each participant retrieves all the rows related to that uuid and changes their event_state as describe below:
• in event of COMMIT:
  • INSERTING(0) -> null
  • LOGICAL_DELETING(1) -> DELETED(-1)
  • DELETING(2) -> row phisically cancelled
  • UPDATE_ARCHIVING(3) -> DELETED(-1)
  • UPDATING(4) -> row phisically cancelled
  • UPDATE_TO_APPLY(5) -> null
• in event of ROLLBACK:
  • INSERTING(0) -> row phisically cancelled
  • LOGICAL_DELETING(1) -> null
  • DELETING(2) -> null
  • UPDATE_ARCHIVING(3) -> null
  • UPDATING(4) -> null
  • UPDATE_TO_APPLY(5) -> row phisically cancelled In order to perform this event_state changes in the right way, the library provides a usefull DAO class to do that avoiding any effort by the developer.

[melloware]

We have already implemented the Quarkus Temporal extension which is a full featured Saga implementation. We are doing a Quarkus Insights video on Monday about it. Cc @rmanibus @tmulle

[melloware]

There are actually two extensions that do Saga already. Quarkus DAPR and Quarkus Temporal.

You can watch the insights to see how it works: https://www.youtube.com/watch?v=XICZxuaeYwI&list=PLsM3ZE5tGAVatO65JIxgskQh-OKoqM4F2

[xian75]

I watched at your video and let me say it was great! I didn't know about Temporal and it's awesome. As you explained in your demo, it looks like very simple developing microservices this way. You just need to define the workflow and the actions to perform step by step. Same way you define the compensations when the request can not be completed succesfully and must be rolledback. I really enjoied the way to define everything: just code. Moreover I was happy to find out that Quarkus Temporal extension and one I'm proposing can work togheter. In fact, my extension focuses on distibuted database transactions trying to solve one of the hardest challenge of microservices: data inconsistency. When I read "Microservices patterns" by Chris Richardson, I asked to myself: what happen if compensation fails? Do I need to compensate the compensation? And if it fails as well? Especially when you have unexpected system errors (bugs). Same questions discussed in https://www.ufried.com/blog/limits_of_saga_pattern/ So I thougth I had to find a way to solve this issue and finally I developed my extension. Basically, it manages compensations automatically leaving data consistent among saga participants. This way you can focus on features development with no efforts for compensations. Moreover you can split your monolithic database and application in a set of micro databases and micro applications letting you scale those ones easier. Finally Temporal extension doesn't exclude my extension and viceversa. On the contrary they can be mixed togheter. For instance, each service of your demo (order, payment, warehouse, etc.) can be realised by my extension in order to scale them easily. I hope some of you will have time to try my extension, even because I already shared a sample you can test just by running that. I keep my fingers crossed.