search

apache / camel-k

Apache Camel K is a lightweight integration platform, born on Kubernetes, with serverless superpowers
https://camel.apache.org/camel-k
Apache License 2.0
867 stars 348 forks source link

Transactions with Camel K #1128

Closed nicolaferraro closed 2 years ago

nicolaferraro commented 4 years ago

Hi @nicolaferraro

We are looking to extend our use of camel-k to include transaction support. I've spent a number of days reading about JTA and technologies such as narayana, and atomikos. I am trying to choose one for fast integration into our prototype architecture.

My goal is to run the transaction manager on kubernetes alongside the camel-k based camel services.

Do you have a recommendation as to a starting point? I'd really like to select and install a transaction manager this week. I dont see a lot of online documentation for running these transaction managers on k8s. Yet I have seen your name on a couple of github projects that seem to relate to this interest. I also know that its of interest to the camel-k and camel team to support the service elements that rely on safe transactions.

Any thoughts would be appreciated.

Originally posted by @davesargrad in https://github.com/apache/camel-k/issues/1059#issuecomment-563243808

nicolaferraro commented 4 years ago

Hi @davesargrad, I've seen your commment on https://github.com/jboss-fuse/spring-boot-camel-xa. Using transactions on Camel K is allowed and hassle-free until they remain local transactions (just use them).

When you need to coordinate multiple transactional resources, such as a JMS system and a database, then it becomes more complex. If you can manage the complexity using an idempotent consumer (it can even be transactional) and JMS retries, you're still safe.

When you need something complex like receive a message, write to db and write an outgoing message in the same transaction, then you need XA. Implementing XA is difficult as your integration needs to be stateful, to run on something like a statefulset and to have a transaction log on a local persistent volume. Scaling out (up and down) integrations is also very complex to achieve.

If you're interested in adding XA support on Camel K, I think the most basic building block is allowing integrations to run on statefulsets with per-pod PV. With that building block, technically an integration can be configured to write the transaction log on the PV. Adding scaling capabilities can come later..

davesargrad commented 4 years ago

@nicolaferraro

Hi. I will ponder your words. I think I understand your thoughts. Best regards.

github-actions[bot] commented 2 years ago

This issue has been automatically marked as stale due to 90 days of inactivity. It will be closed if no further activity occurs within 15 days. If you think that’s incorrect or the issue should never stale, please simply write any comment. Thanks for your contributions!