SGrondin / bottleneck

Job scheduler and rate limiter, supports Clustering
MIT License
1.82k stars 75 forks source link

bottleneck lost tracing data when using reservoir options #211

Open lingjin666 opened 1 year ago

lingjin666 commented 1 year ago

Here is my code

import Bottleneck from 'bottleneck';
import { AsyncLocalStorage } from 'async_hooks';

const asyncLocalStorage = new AsyncLocalStorage();

const bottleneck = new Bottleneck({
    reservoir: 3,
    reservoirRefreshInterval: 5000,
    reservoirRefreshAmount: 3,

    maxConcurrent: 1,
    minTime: 1000,
});

const init = async () => {
    const num = 1;
    for (let i = 0; i < 100; i++) {
        await asyncLocalStorage.run(i, async () => {
            await bottleneck.schedule(() => {
                return asyncFun();
            });
        });
    }
};

async function asyncFun() {
    console.log(asyncLocalStorage.getStore());
}

init();

output:

0
1
2
undefined
4
5
undefined
7
8
undefined
10
11
.......

Normally, it will print from 0 to 99. But when meet 3/6/9...., it prints undefined. So why ?

$ node -v    
v18.12.1
abou7mied commented 3 weeks ago

I'm encountering the same issue, I ended up with a workaround by re-assigning the store data within the job function:

import Bottleneck from 'bottleneck'
import { AsyncLocalStorage } from 'async_hooks'

const asyncLocalStorage = new AsyncLocalStorage()

const bottleneck = new Bottleneck({
  reservoir: 3,
  reservoirRefreshInterval: 5000,
  reservoirRefreshAmount: 3,

  maxConcurrent: 1,
  minTime: 1000,
})

const init = async () => {
  const num = 1
  for (let i = 0; i < 100; i++) {
    await asyncLocalStorage.run(i, async () => {
      const store = asyncLocalStorage.getStore() // <---- read current store
      await bottleneck.schedule(() => {
        asyncLocalStorage.enterWith(store) // <---- enters with it
        return asyncFun()
      })
    })
  }
}

async function asyncFun() {
  console.log(asyncLocalStorage.getStore())
}

init()

output:

0
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
...

It's not the best, but it does the job.