camunda-community-hub / zeebe-client-node-js

Node.js client library for Zeebe Microservices Orchestration Engine
https://camunda-community-hub.github.io/zeebe-client-node-js/
Apache License 2.0
152 stars 38 forks source link

make BatchWorker batch interrupt safe #179

Closed jwulf closed 4 years ago

jwulf commented 4 years ago

Fixes #177

Proposed Changes

jetdream commented 4 years ago

How is this fixes #177? In case of receiving data on closing stage it still loses jobs.

To reliably fix this I think the connection must be closed by server, which has to be sure that no jobs lost when closing connection. At the moment there is no explicit way to ask server to close long polling connection, thus closing can take as long as long polling timeout.

Or might be worth considering introducing acknowledgement-based approach when job is moved to ACTIVATED state only after being acknowledged by job handler.

jwulf commented 4 years ago

How is this fixes #177? In case of receiving data on closing stage it still loses jobs.

Closing stage is specifically for application shut down. To trigger that case, you have to call ZBClient.close() or ZBBatchWorker.close().

I think the source of your issue is jobs streaming to the worker while a Batch is executing. It looks like there was a possible race condition: the batch processing uses a copy of the array of jobs. So the batch processor could be invoked with a copy of the batch, and then the batch gets updated.

I replaced the batched jobs array and the call with copy operation with a queue implemented using a mutable array.