networknt / light-eventuate-4j

An eventual consistency framework based on Event Sourcing and CQRS on top of light-4j and Kafka
Apache License 2.0
59 stars 20 forks source link

[discussion] Microservices are "just" highly scalable BPM engines/processes #36

Open archenroot opened 6 years ago

archenroot commented 6 years ago

Hi guys,

I have another interesting point to discuss. I hope you will agree on the subject of this issue. So we left BPM engines because they were just fat slow beasts missing all cool things we can do in pure event driven decoupled architectures.

No imagine that you can merge these 2 worlds back together. Having the capability of implementing the process in BPMN language, which si still superior tool and at the same moment you can split process and scale each components independently, supporting Sagas, Event Sourcing, CQRS, Auditing, etc.

Look at this promising project: https://zeebe.io/

I think they have something fundamentally wrong -> they are still trying to work on top of service layer being Orchestrating... which is against event driven core concept :-), but I think it could be promising possible merge in some far future...

GavinChenYan commented 6 years ago

@archenroot , this is a good topic.

BPM engines is fat and slow. But use event driven architecture to handle workflow may have massive volume of event message for complex workflow. Merge these two together could be good idea.

archenroot commented 6 years ago

@chenyan71 - yeah and always, will be :-))

Listen, I discussed this topic with one geek on bpm engines and microservices and he actually came with some interesting ideas to discover which I in generally agreed or can be applicable in some scenarios, the article and discussion can be seen here: https://blog.bernd-ruecker.com/saga-how-to-implement-complex-business-transactions-without-two-phase-commit-e00aa41a1b1b

Against Zeebe, which is like global orchestrator, which I didn't like at all, he is actually implementing BPM process on one of the services (owner of the process) and especially here he describe merging of both choreography and orchestration concepts into one: https://blog.bernd-ruecker.com/why-service-collaboration-needs-choreography-and-orchestration-239c4f9700fa

stevehu commented 6 years ago

@archenroot @chenyan71 This is a very interesting topic and there are a lot of good ideas bouncing around in this thread. I have been working on workflow for a long time and very familiar with BPMN and the alike. In my opinion, it is not the answer for our case. Also, we need to separate transaction orchestration from business process orchestration as they are focusing on different life-span. For distributed transaction, it lasts up to several minutes; however, for business processes, it can last months or years. In my roadmap, I have two distinct products to cover each of them. light-saga-4j is purely for distributed transactions and light-workflow-4j is used for business processing. For the workflow, I don' t recommend to use BPEL as it is very heavy and slow. I am using a state machine engine to implement the workflow and it is still working in progress.

archenroot commented 6 years ago

@stevehu nice, I am going to look at workflow-4j now..... Totally agree that for high-performance BPEL is not an option. I like bernd's approach with per service workflows without any central orchestrator, here he provides fully working example, I will give it a try and compare: https://github.com/flowing/flowing-retail

archenroot commented 6 years ago

I think your workflow-4j is "just" high-performance micro-bpm engine, but what is more important is the concept of binding the process into owner service, sou have workflow/bpm, but still at scale. His drawings on that github projects are really nice :-) worth words

archenroot commented 6 years ago

@stevehu - also found quite interesting stuff - http://copper-engine.org/