phillbaker / digitalocean-node

Unofficial node client for the v2 DigitalOcean API
http://phillbaker.github.io/digitalocean-node
MIT License
67 stars 16 forks source link

Throttle requests #37

Closed jorisw closed 2 years ago

jorisw commented 2 years ago

When my code removes a bunch of droplets at the same time, I get 429 errors like this one:

[DigitalOceanError: Too many requests] {
  statusCode: 429,
  headers: {
    date: 'Wed, 17 Nov 2021 07:30:51 GMT',
    'content-type': 'application/json',
    'content-length': '60',
    connection: 'close',
    'ratelimit-limit': '5000',
    'ratelimit-remaining': '0',
    'ratelimit-reset': '1637130654',
    'x-gateway': 'Edge-Gateway',
    'x-request-id': 'ac391732-4926-46fc-8394-9cf1a2ea7e0a',
    'x-response-from': 'Edge-Gateway',
    'cf-cache-status': 'DYNAMIC',
    'expect-ct': 'max-age=604800, report-uri="https://report-uri.cloudflare.com/cdn-cgi/beacon/expect-ct"',
    server: 'cloudflare',
    'cf-ray': '6af7418f1f4d00a3-AMS'
  },
  body: { id: 'Too Many Requests', message: 'Too many requests' }
}

Some client libraries like ccxt (a crypto currency exchange client) have built-in throttle queues that adhere to rate limiting, preventing 429 errors. Suggestion to add this to this client library.

jorisw commented 2 years ago

promise-ratelimit actually works well for this.

phillbaker commented 2 years ago

Thanks, mind sharing an example?

On Sat, Dec 11, 2021 at 10:09 AM Joris W @.***> wrote:

promise-ratelimit https://www.npmjs.com/package/promise-ratelimit actually works well for this.

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/phillbaker/digitalocean-node/issues/37#issuecomment-991676471, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAAXCKMOOXQQKLWLZL3NZTDUQNSS3ANCNFSM5ILU4LHQ . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.

jorisw commented 2 years ago

As simple as:

const digitalOcean = require("digitalocean");
const promiseRateLimit = require("promise-ratelimit");

const digitalOceanClient = digitalOcean.client(process.env.DIGITALOCEAN_TOKEN);
const throttleRequests = promiseRateLimit(1000);

try {
  // Make sure 1000ms have passed since last request
  await throttleRequests();

  // List droplets   
  const apiDroplets = await digitalOceanClient.droplets.list(1, 200);

} catch (e) {
  console.error(e);
}