search

camunda-community-hub / zeebe-client-node-js

Node.js client library for Zeebe Microservices Orchestration Engine
https://camunda-community-hub.github.io/zeebe-client-node-js/
Apache License 2.0
152 stars 38 forks source link

Workers stop polling until a restart #309

Closed nhomble closed 1 year ago

nhomble commented 1 year ago

Issue tracker is ONLY used for reporting bugs. New features and questions should be discussed on our slack channel.

https://camunda.com/blog/2020/05/zeebe-node-0-23/#connection-error-debouncing

Expected Behavior

Workers should refresh their connection with the broker if lost.

Current Behavior

Workers stop polling. It's also unclear from the INFO+ logs what the issue could be (we are in the process of reenabling debug logs to triage this issue better).

Possible Solution

Steps to Reproduce

  1. running library at 8.1.8
  2. running workers against camunda saas
  3. within a couple hours, we see process instances build up

Context (Environment)

Process instances are left pending.

Detailed Description

Possible Implementation

nhomble commented 1 year ago

After we added DEBUG=grpc,oauth

we see the workers suddenly stop processing while completing work

2023-03-29T20:17:24.821Z oauth [OAuth] Using cached token from memory...
2023-03-29T20:17:24.821Z grpc Calling completeJobSync...

unfortunately there are no other library logs

nhomble commented 1 year ago

Haven't been able to reproduce.