SGrondin / bottleneck

Job scheduler and rate limiter, supports Clustering
MIT License
1.81k stars 74 forks source link

Job counts(limiter.counts()) cannot be retrieved #186

Open bhrigushr opened 3 years ago

bhrigushr commented 3 years ago

I am using Bottleneck to rate limit API requests and it's in distributed env(multiple servers) I want to report the status of jobs in limiter.

I tried to get status using the following:-

const group = new Bottleneck.Group({ clientOptions: redisConfig, datastore: 'ioredis', id:'someId' });
const limiters = group.limiters();
const status = limiters.map((el) => {
        return {
             key: el.key,
            ...el.limiter.counts(),
          },
        };
      }),

Getting status as an empty array

Also tried to get status using:-

 const limiter = new Bottleneck({
      clientOptions: redisConfig,
      datastore: 'ioredis',
      id: 'someLimiterId',
    });
    const status = limiter.counts();

Get status as:-

{
  "RECEIVED": 0,
  "QUEUED": 0,
  "RUNNING": 0,
  "EXECUTING": 0
}

even though there are jobs for limiter

I found that if I call the API limiter.counts() with the same instance object of limiter or group which is rate-limiting my API calls then I get the correct result

I want to retrieve the status of jobs in the different server which is not running those jobs

I am doing something wrong? Or Please advise how can I achieve the above functionality

duinness commented 1 year ago

The Github rollercoaster. You search the issues and find one with your EXACT problem 😄 and scroll down to find that there has been no response in a year and a half 😭 .

I know it is a long shot but, @bhrigushr, did you ever find a work-around for this?

bhrigushr commented 1 year ago

@duinness I used individual functions to get stats like clusterQueued, running

duinness commented 1 year ago

@bhrigushr that was the exact conclusion I came to yesterday 👍

In case someone runs into this two years from now...

I ended up using node-redis and connected directly to my shared redis instance.

export const getBottleneckWaitingQueueCount = async () => { const waitingQueues = await client.hVals('b_service_name:local::queue_name_client_num_queued'); const sum = waitingQueues.reduce((partialSum, a) => partialSum + Number(a), 0); return sum; };