Polling is not correct under load

[jwulf]

Given this scenario:

Activate ten jobs at a time, with a timeout of 60 seconds, and rate limit their completion to one every five seconds.

I would expect to see a repoll every n seconds, where n is something related to the capacity threshold for re-polling, but I see this:

ws    | 06:47:49.685 | zeebe |  [email:send] INFO: Worker at max capacity - email:send has 10, a capacity of 10, and a minimum job batch size of 0.
ws    | 06:47:49.985 | zeebe |  [email:send] INFO: Worker at max capacity - email:send has 10, a capacity of 10, and a minimum job batch size of 0.
ws    | 06:47:50.284 | zeebe |  [email:send] INFO: Worker at max capacity - email:send has 10, a capacity of 10, and a minimum job batch size of 0.
ws    | 06:47:50.584 | zeebe |  [email:send] INFO: Worker at max capacity - email:send has 10, a capacity of 10, and a minimum job batch size of 0.
ws    | 06:47:50.885 | zeebe |  [email:send] INFO: Worker at max capacity - email:send has 10, a capacity of 10, and a minimum job batch size of 0.
ws    | 06:47:51.184 | zeebe |  [email:send] INFO: Worker at max capacity - email:send has 10, a capacity of 10, and a minimum job batch size of 0.
ws    | 06:47:51.484 | zeebe |  [email:send] INFO: Worker at max capacity - email:send has 10, a capacity of 10, and a minimum job batch size of 0.
ws    | 06:47:51.785 | zeebe |  [email:send] INFO: Worker at max capacity - email:send has 10, a capacity of 10, and a minimum job batch size of 0.
ws    | 06:47:52.086 | zeebe |  [email:send] INFO: Worker at max capacity - email:send has 10, a capacity of 10, and a minimum job batch size of 0.
ws    | 06:47:52.387 | zeebe |  [email:send] INFO: Worker at max capacity - email:send has 10, a capacity of 10, and a minimum job batch size of 0.
ws    | 06:47:52.686 | zeebe |  [email:send] INFO: Worker at max capacity - email:send has 10, a capacity of 10, and a minimum job batch size of 0.
ws    | 06:47:52.987 | zeebe |  [email:send] INFO: Worker at max capacity - email:send has 10, a capacity of 10, and a minimum job batch size of 0.
ws    | 06:47:53.287 | zeebe |  [email:send] INFO: Worker at max capacity - email:send has 10, a capacity of 10, and a minimum job batch size of 0.
ws    | 06:47:53.588 | zeebe |  [email:send] INFO: Worker at max capacity - email:send has 10, a capacity of 10, and a minimum job batch size of 0.

[jwulf]

The polling should work like this:

• By default, the activation threshold is 30% of the maximum jobs (example 3 for a max of 10 jobs)
• After polling, if the threshold is not exceeded (there are less than 7 jobs activated), poll again.
• If the threshold is exceeded, wait until jobs are drained. Each time a job is drained, check if the threshold if exceeded. If it is not exceeded, poll again.
• Deal with the long poll timeout.

[jwulf]

At the moment, it looks like it is repolling every 300ms. It should block, and repoll when capacity is released using the emitter in drainOne(). Otherwise it will saturate the event loop with unnecessary activity.

[jwulf]

This is literally the best idea evah

[MaximMonin]

300 ms is default poll Interval. I am using 5ms for stress testing for worker: pollInterval: Duration.seconds.of (0.005) I think client should limit max threads, to max jobs

[jwulf]

The Node client is single-threaded. The event loop has a thread, and the operating system threads handle async I/O.