Minimum retry time seems to be 5 seconds

[kibertoad]

Retry timeout parameter doesn't seem to matter unless it's more than 5 seconds.

If I set my backoff strategy to exponential/1, my test (which does retry once) finishes in 5 seconds 220 msecs. If I set my backoff strategy to exponential/1000, it finishes within approximately same time. Same for exponential/4000

But if I set it to exponential/6000, it finishes in 6 seconds 220 msecs, and execution time keeps growing accordingly if I increase it even further.

I can understand why this makes sense for production, but for tests burning 5 seconds for each test that does a retry is unacceptable. Would it be possible to support setting very low retry delays?

Reproduction

Any simple job that throws on execution will do, it is not related to logic being executed. Logs indicate that Bullmq doesn't even attempt to retry the failed job until 5 seconds pass, e. g.:

[2023-11-04 21:25:18.944 +0200] INFO: Finished processing test_job (error) (2b74c8e9-bbcc-496c-95f8-cdbaf4f818c1)
[2023-11-04 21:25:23.990 +0200] INFO: Started processing test_job (2c76a048-9266-4d3f-90b5-0d7d6c6f5d06)

Versions:

• bullmqpro 6.6.1
• bullmq 4.12.3

[manast]

5 seconds seems like a suspicious value, as it is the standard blocking timeout, maybe we are not properly calculating the block timeout based on the backoff value, @roggervalf what do you think?

[roggervalf]

hi @kibertoad, I cannot replicate your case, it'll be good if you can provide a test case to replicate it

[roggervalf]

btw I'm adding one extra test case using exponential backoff and it's not taking more than 2 seconds https://github.com/taskforcesh/bullmq/pull/2266

[kibertoad]

@roggervalf sure, I'll try to create a repro tonight.

[kibertoad]

Sorry for the delay. I can still reproduce the issue with the latest bullmq. Working on a repro now.

[kibertoad]

OK, so it previously went away for us after we set the following for our code:

if (this.config.isTest && typeof preparedOptions.backoff === 'object') {
            preparedOptions.backoff.delay = 0
        }

However, backoff.delay seems to be ignored when concurrency of the queue is larger than 1, and that resets the delay back to 5 seconds in all test cases for whatever reason, regardless of what configuration there is.

I'll try to create a repro again this time, it should be easier to reproduce.

[kibertoad]

@manast Actually, this seems to be a regression from the past: https://github.com/taskforcesh/bullmq/issues/1505

We are seeing the same behaviour with BullMQ 5.4.2/BullMQ Pro 7.0.0

This is for a non-recurring job with concurrency > 1, attempts = 3 and backoff.delay = 0