jhurliman / node-rate-limiter

A generic rate limiter for node.js. Useful for API clients, web crawling, or other tasks that need to be throttled
MIT License
1.51k stars 135 forks source link

Feature req: chainable limiters, configuration functions and Promises #20

Closed amyspark closed 4 years ago

amyspark commented 9 years ago

Hi, I'd love for this library to have the following functionality, as it would help me a lot in the app I'm developing:

Would that be possible? If not, please let me know and I'll see if I can add them.

Thanks!

rafaelvanderlei commented 6 years ago

Hi, I'm relatively new with js libraries, thus I'm not familiar enough to write Promises, but I've seen how awesome they are, rather than using callbacks and when I needed to limit my requests, I found this library but unfortunately there is no built-in support to promises.

That being said, in the search for using node-rate-limiter with Promise support, I found this issue and thanks to your tip of using bluebird (which I had never heard about - it might give you an idea of how newbie I am in the js world), I managed to use node-rate-limiter with promises with the following code (I don't know if its the easiest way, but hey.. it's working for me):

var Promise = require('bluebird')
var RateLimiter = require('limiter').RateLimiter
var limiter = new RateLimiter(1, 'second')
var limiterAsync = Promise.promisifyAll(limiter)

var rp = require('request-promise')
var promises = []

console.log(new Date(), 'starting...')
for (let i = 0; i < 10; i++) {
  var promise = limiterAsync.removeTokensAsync(1)
    .then(() => rp('https://jsonplaceholder.typicode.com/todos/1')
      .then(response => ({ time: new Date(), data: JSON.parse(response) })))
    .catch(err => console.log(err))
  promises.push(promise)
}
console.log(new Date(), 'all requests sent to limiter...', promises.length)

Promise.all(promises)
  .then(values => {
    console.log(new Date(), 'all promises responded')
    values.forEach(value => console.log(value))
  })

With the code above, for test purposes, the intention is to fire 10 requests, being one request per second (throttling controlled by node-rate-limiter), but I want a promise to be returned right away..

Running the code, the messages "starting..." and "all requests sent to limiter... 10" are printed almost instantly, showing the 10 promises were created asynchronously. Then, 10 seconds later, the message "all promises responded" is printed and for each value, the value of the time property is 1 second ahead of the previous value, showing the throttling was respected and that 1 request was sent per second... the best being that I was able to get a promise from the call to removeTokensAsync, created by bluebird (which wasn't possible with the original version of RateLimiter#removeTokens)

I don't know if this statement can help someone make this a built-in feature to node-rate-limiter, but at least it might serve as a workaround for those who want to work with node-rate-limiter and promises altogether.

rafaelvanderlei commented 6 years ago

To make things more organized in my project, I've created a simple internal module:

rateLimiterAsync.js

import Promise from 'bluebird'
import { RateLimiter } from 'limiter'

export const createLimiterAsync = (tokensPerInterval, interval, fireImmediately) => {
  return Promise.promisifyAll(new RateLimiter(tokensPerInterval, interval, fireImmediately))
}

and when I need to use node-rate-limiter with promise, I just do:

import { createLimiterAsync } from '../../lib/rateLimiterAsync'

const limiterAsync = createLimiterAsync(1, 'second')
limiterAsync.removeTokensAsync(1)
    .then(() => rp('https://jsonplaceholder.typicode.com/todos/1')
      .then(response => ({ time: new Date(), data: JSON.parse(response) })))
    .catch(err => console.log(err))