Long delay in Bottleneck.schedule execution in a distributed env

I have been using Bottleneck to rate-limit external API calls from the system in a distributed environment. This was perfect till now but once the number of servers increased I see a considerable delay in API calls.

API Rate limit -> 500 requests/min per tenant or account with max concurrent 10 requests For each tenant/account we make ~1.5k requests During testing, we were pulling data for 30 tenants/accounts Total API call -> 45k(30*1500)

When running 50 servers -> It took ~5 mins to make 45k API calls

When running 100 servers -> It took ~15 mins to make 45k API calls

My code looks like

const reqsPerMin= 500;
const params = {
      id:'some_unique_id',
      datastore: 'ioredis',
      maxConcurrent: 10,
      minTime: Math.ceil((60 * 1000) / reqsPerMin),
      clientOptions: redisConfig,
      reservoir: reqsPerMin,
      reservoirRefreshInterval: 60 * 1000,
      reservoirRefreshAmount: reqsPerMin,
      highWater: reqsPerMin,
      strategy: 'OVERFLOW',
    };
 const limiterGroup = new Bottleneck.Group(params);

 const requests = [...]; // requests to make to external service
await Promise.all((request) => {
const limiter = limiterGroup.key(request.tenantId);
  return limiter.schedule({ priority: 1 }, () => {
    // make API call
    return sendRequest(request);
  });
});

SGrondin / bottleneck

Long delay in Bottleneck.schedule execution in a distributed env #214