SGrondin / bottleneck

Job scheduler and rate limiter, supports Clustering
MIT License
1.77k stars 73 forks source link

intervals not cleared in Deno #225

Open michielbdejong opened 1 week ago

michielbdejong commented 1 week ago

Deno.test("Bottleneck", async () => { const retryLimiter = new Bottleneck(); retryLimiter.schedule(async () => {}); });

Deno.test("BottleneckLight", async () => { const retryLimiter = new BottleneckLight(); retryLimiter.schedule(async () => {}); });

* Run:
```sh
$ deno test --trace-leaks test.ts
running 2 tests from ./test.ts
Bottleneck ... FAILED (4ms)
BottleneckLight ... FAILED (2ms)

 ERRORS 

Bottleneck => ./test.ts:4:6
error: Leaks detected:
  - A timer was started in this test, but never completed. This is often caused by not calling `clearTimeout`. The operation was started here:
    at Object.queueUserTimer (ext:core/01_core.js:738:9)
    at setTimeout (ext:deno_web/02_timers.js:61:15)
    at Timeout.<computed> (ext:deno_node/internal/timers.mjs:68:7)
    at new Timeout (ext:deno_node/internal/timers.mjs:55:37)
    at setTimeout (node:timers:12:10)
    at Bottleneck._run (file:///Users/michiel/Library/Caches/deno/npm/registry.npmjs.org/bottleneck/2.19.5/lib/Bottleneck.js:198:18)
    at file:///Users/michiel/Library/Caches/deno/npm/registry.npmjs.org/bottleneck/2.19.5/lib/Bottleneck.js:256:18
    at eventLoopTick (ext:core/01_core.js:207:9)

BottleneckLight => ./test.ts:9:6
error: Leaks detected:
  - A timer was started before the test, but completed during the test. Intervals and timers should not complete in a test if they were not started in that test. This is often caused by not calling `clearTimeout`. The operation was started here:
    at Object.queueUserTimer (ext:core/01_core.js:738:9)
    at setTimeout (ext:deno_web/02_timers.js:61:15)
    at Timeout.<computed> (ext:deno_node/internal/timers.mjs:68:7)
    at new Timeout (ext:deno_node/internal/timers.mjs:55:37)
    at setTimeout (node:timers:12:10)
    at file:///Users/michiel/Library/Caches/deno/npm/registry.npmjs.org/bottleneck/2.19.5/lib/LocalDatastore.js:81:14
    at new Promise (<anonymous>)
    at LocalDatastore.yieldLoop (file:///Users/michiel/Library/Caches/deno/npm/registry.npmjs.org/bottleneck/2.19.5/lib/LocalDatastore.js:80:12)
    at file:///Users/michiel/Library/Caches/deno/npm/registry.npmjs.org/bottleneck/2.19.5/lib/LocalDatastore.js:274:21
    at Generator.next (<anonymous>)
  - A timer was started in this test, but never completed. This is often caused by not calling `clearTimeout`. The operation was started here:
    at Object.queueUserTimer (ext:core/01_core.js:738:9)
    at setTimeout (ext:deno_web/02_timers.js:61:15)
    at Timeout.<computed> (ext:deno_node/internal/timers.mjs:68:7)
    at new Timeout (ext:deno_node/internal/timers.mjs:55:37)
    at setTimeout (node:timers:12:10)
    at Bottleneck._run (file:///Users/michiel/Library/Caches/deno/npm/registry.npmjs.org/bottleneck/2.19.5/light.js:1201:19)
    at file:///Users/michiel/Library/Caches/deno/npm/registry.npmjs.org/bottleneck/2.19.5/light.js:1236:19
    at eventLoopTick (ext:core/01_core.js:207:9)
    at async file:///Users/michiel/Library/Caches/deno/npm/registry.npmjs.org/bottleneck/2.19.5/light.js:773:24
    at async Sync._tryToRun (file:///Users/michiel/Library/Caches/deno/npm/registry.npmjs.org/bottleneck/2.19.5/light.js:771:14)

 FAILURES 

Bottleneck => ./test.ts:4:6
BottleneckLight => ./test.ts:9:6

FAILED | 0 passed | 2 failed (11ms)

error: Test failed
michielbdejong commented 1 week ago

The following deno test passes unless you comment out one of the commented lines:

import BottleneckLight from "npm:bottleneck/light.js";

Deno.test("BottleneckLight", async () => {
  const limiter = new BottleneckLight();
  // limiter.schedule(() => { console.log('hi'); });
  // await new Promise((resolve) => limiter.schedule(() => { resolve(null); }));
  // await new Promise((resolve) => limiter.schedule(() => { resolve(null); console.log('hi'); }));
  await new Promise((resolve) => limiter.schedule(() => { console.log('hi'); resolve(null); }));
});
michielbdejong commented 1 week ago

Even with console.log('hi'); resolve(null); in the schedule callback it fails intermittently, about 20% of the time on my laptop.

michielbdejong commented 1 week ago

A pretty consistent workaround seems to be the following:


import BottleneckLight from "npm:bottleneck/light.js";

Deno.test("BottleneckLight", async () => {
  const limiter = new BottleneckLight();
  await new Promise((resolve) => limiter.schedule(() => {
    resolve(null);
  }));
  // work around https://github.com/SGrondin/bottleneck/issues/225
  await new Promise(r => setTimeout(r, 0));
});