SpiderStrategies / node-tweet-stream

Node twitter module to hook into the public filter streaming, seamlessly updating the tracking keywords.
210 stars 43 forks source link

tweet.on(error) doesn't handle Exceeded Rate Limit #39

Open kurisubrooks opened 8 years ago

kurisubrooks commented 8 years ago
tweet.on('error', function(error) {
    console.log('Error: ' + error);
});

I use the above code, but for some reason, lately i'm getting Exceeded Rate Limit crashes. Shouldn't this stop it and just log it to console instead, rather than ending the program?

nathanbowser commented 8 years ago

Yea, sounds like it. PR welcome.

tracycollins commented 6 years ago

Looks like this is still a problem? I'm getting "Exceeded twitter rate limit" errors which crash the program:

Error: Exceeded twitter rate limit at Twitter.networkBackoff (/home/.../node_modules/node-tweet-stream/lib/twitter.js:9:13) at Twitter. (/home/.../node_modules/node-tweet-stream/lib/twitter.js:327:13) at Request.emit (events.js:182:13) at Request.EventEmitter.emit (domain.js:442:20) at Request.onRequestError (/home/.../node_modules/request/request.js:877:8) at ClientRequest.emit (events.js:182:13) at ClientRequest.EventEmitter.emit (domain.js:442:20) at TLSSocket.socketErrorListener (_http_client.js:382:9) at TLSSocket.emit (events.js:182:13) at TLSSocket.EventEmitter.emit (domain.js:442:20) at emitErrorNT (internal/streams/destroy.js:82:8) at emitErrorAndCloseNT (internal/streams/destroy.js:50:3) at process._tickCallback (internal/process/next_tick.js:63:19)

Looking at node-tweet-stream/lib/twitter.js, the crash is caused in the "backoff" function when the "current" backoff value exceeds the "max", which in my case max=16s for network:

function backoff (current, max, step, _value) {
  return function () {
    if ((_value = current) > max) {
      throw new Error('Exceeded twitter rate limit')
    }
    current = step(current)
    return _value
  }

}

which is called by:

Twitter.prototype.backoffs = function () {
  // Network hiccup, try every 250 seconds
  this.networkBackoff = backoff(0, 16 * 1000, function (x) { return x + 250 })
  // Rate limited. Try exponetially starting at 5 seconds
  this.httpBackoff = backoff(5 * 1000, 320 * 1000, function (x) { return x * 2 })
  // Rate limited. Try exponetially starting at a minute
  this.rateBackoff = backoff(60 * 1000, Infinity, function (x) { return x * 2 })
}

Maybe I'm missing something, but why crash the program for this type of error?