search

paed01 / bpmn-engine

BPMN 2.0 execution engine. Open source javascript workflow engine.
MIT License
884 stars 167 forks source link

Can it be scalable when its deployed in microservices? #153

Closed BaluprasathSBP closed 2 years ago

BaluprasathSBP commented 2 years ago

Hi,

We will gonna be deploying the embedded bpmn-engine application as a microservices in n pods. I believe that instance of an Engine is specific to a pod in this case. Would this be still possible to scale it horizontally and share the same state between pods?

paed01 commented 2 years ago

Yes, I have done it for onify.

You need to save the state somewhere accessible between pods, of course. I suggest you start by saving state when an activity is emitting wait or timer. Then implement some sort of signalling functionality to resume from a saved state.

BaluprasathSBP commented 2 years ago

Thanks for your response.

I feel the suggested approach would work only if it is a pull-based(which is client requests for what next), then the service would look in to the saved state and triggers another activity.

Considering the timers(intermediate or looping) where the server should trigger the flow of activities. Since the Engine is embedded in two pods, how could it be made as an one orchestrator to ensure that is not triggered twice at a same time. please advise.

should it be done using orchestrator server?

paed01 commented 2 years ago

Have you considered leader election?

BaluprasathSBP commented 2 years ago

Thanks @paed01. Looking into it, But if we use Leader election method then, only one pod would be running multiple engines. Suppose, few thousand instances of a workflow needs to be executed at a same time, then only one pod as a leader would be heavy load to it.