search

DonDebonair / slack-machine

A simple, yet powerful and extendable Slack bot
https://dondebonair.github.io/slack-machine/
MIT License
760 stars 54 forks source link

Rate Limit Handling for Resilient Slack API Interaction #1192

Closed jasonwbarnett closed 1 week ago

jasonwbarnett commented 1 week ago

Summary

This PR introduces asynchronous rate limit handling to improve Slack Machine’s resilience when interacting with the Slack API. By incorporating error management for SlackApiError and implementing a retry mechanism, this update aims to minimize disruptions during high-volume API requests and ensure the bot doesn't fail to start under rate-limiting conditions, such as when there are thousands of users and/or channels.

Fixes https://github.com/DonDebonair/slack-machine/issues/975

Changes

  1. Rate Limit Management: Adds handling for SlackApiError with specific checks for rate-limiting errors, allowing the bot to pause and retry when limits are hit.
  2. Pagination and Caching: Introduces build_paginated_cache, a method for managing paginated API responses, enabling smoother and more efficient data handling.

Motivation

Slack’s API rate limits can cause disruptions in bot functionality, especially in high-demand environments. This PR adds rate limit handling to Slack Machine, providing a smoother, more resilient user experience.

Testing

All changes have been tested locally under various rate-limit scenarios to confirm proper error handling and retry logic.

jasonwbarnett commented 1 week ago

It seemed like https://github.com/DonDebonair/slack-machine/pull/1115 is dead so I'm trying to resurrect since it's a fairly straight forward change that is needed for larger Slack deployments.

DonDebonair commented 1 week ago

Great addition! I left one comment with a proposed alternative approach to reusing the retry logic. Aside from that, both ruff and mypy complain 😅

jasonwbarnett commented 1 week ago

Great addition! I left one comment with a proposed alternative approach to reusing the retry logic. Aside from that, both ruff and mypy complain 😅

Thanks for quick review. I'll take a loop on these items today/tomorrow.

jasonwbarnett commented 1 week ago

@DonDebonair Ready for 👀

DonDebonair commented 1 week ago

MyPy is not happy and I think the code can be simplified :) See my comment.

Apologies for the back-and-forth, I tend to be nitpicky about code readability.

jasonwbarnett commented 1 week ago

MyPy is not happy and I think the code can be simplified :) See my comment.

Apologies for the back-and-forth, I tend to be nitpicky about code readability.

No problem. I am picky about readability as well.