scrapinghub / slackbot

A chat bot for Slack (https://slack.com).
MIT License
1.26k stars 394 forks source link

Respect the rate limit of slack API #39

Open lins05 opened 8 years ago

lins05 commented 8 years ago

Slack imposes "at-most 1req/sec" rate limit on all its api (both web api and RTM api). When it's exceeded the application gets http 429 (Too many requests). The slackbot core should control the rate of sending requests to make sure the rate limit is not exceeded.

https://api.slack.com/docs/rate-limits

alphon-sh commented 7 years ago

Hey :) Any news on this subject ? Got a lot of disconnection on my bots :/

Something like this in the message.send_webapi function must work but I'm having trouble getting the json response from slack.

if response.status_code == 429: print('Rate limited! Please wait.') time.sleep(int(response.headers['retry-after']))

I'm a beginner :)

alphon-sh commented 7 years ago

In slackclient.py

def rtm_connect(self):
        obj = self.webapi.rtm.start()
        reply = obj.body
        if not obj.successful:  #Would be better to check obj.error = 429
            time.sleep(60)
        time.sleep(1)
        self.parse_slack_login_data(reply)

This could work, any ideas to improve it ?

Or add the headers in the Response object coming from Slacker, so that we can check if there is a retry-after ...

jtatum commented 7 years ago

@alphon-sh Just curious - why are you using send_webapi? You might have some luck changing over to message.send or message.reply...

Rate limiting is tough. We really do need to work on it, but the solution will be a little more complicated than updating or even decorating each method. We'll probably have to put sends into a queue and have the queue manager do exponential backoffs on commands.

orgito commented 7 years ago

My naive workaround:

from time import time, sleep
from slackclient import SlackClient

class WellBehavedSlackClient(SlackClient):
    '''Slack client with rate limit'''

    def __init__(self, token, proxies=None, ratelimit=1.0):
        super().__init__(token, proxies)
        self.ratelimit = ratelimit
        self.last_invoked = time() - ratelimit

    def api_call(self, method, timeout=None, **kwargs):
        while True:
            now = time()
            if (now - self.last_invoked) >= self.ratelimit:
                result = super().api_call(method, timeout=timeout, **kwargs)
                self.last_invoked = time()
                return result
            else:
                sleep(self.ratelimit - (now - self.last_invoked))

client = WellBehavedSlackClient('my-token')

Now the client will wait at least ratelimit seconds between calls. Works for my use cases. YMMV