Techniques or approaches for avoiding Slack API rate limit

erkand-imeri commented 1 year ago

Hitting Slack API rate limits while using Slack Bolt with Node.js and TypeScript

Description:

I'm using the Slack Bolt framework to create a Slack app that allows dispatchers to communicate with customer service agents in Zendesk. However, I'm constantly hitting Slack API rate limits when posting messages and fetching thread replies. I need help optimizing my code to reduce the number of API calls and avoid hitting the rate limits.

Reproducible in:

The Slack SDK version

"@slack/bolt": "^3.11.0",
"@slack/web-api": "^6.7.1",

Node.js runtime version

Dockerfile: node:14-alpine

OS info

Kubernetes, Docker.

Steps to reproduce:

/**
 * Get the parent message for a given thread message in a given channel.
 *
 * @param slackApp
 * @param channelId
 * @param threadId
 * @param slackToken
 */
export const getParentMessage = async ({
  slackApp,
  channelId,
  threadId,
  slackToken,
}: {
  slackApp: App;
  channelId: string;
  threadId: string;
  slackToken?: string;
}): Promise<ParentTicketMessage> => {
  const result: WebAPICallResult & {
    messages?: Array<GenericMessageEvent>;
  } = await slackApiChannelIdWrapper((channelId) => {
    return slackApp.client.conversations.replies({
      token: slackToken || slack.token,
      channel: channelId,
      ts: threadId,
      limit: 10,
      inclusive: true,
      oldest: '1',
    });
  }, channelId);

  if (!result.messages) {
    return {} as ParentTicketMessage;
  }

  const parentMessage = result.messages[0] || ({} as GenericMessageEvent);
  const parentAttachments: MessageAttachment[] =
    parentMessage.attachments || ({} as MessageAttachment[]);
  const parentBlocks = parentAttachments[0].blocks;
  const channelCountryCode = getCountryCodeByChannelId(String(channelId));

  const getTicketData = (): TicketCreate => ({
    ...extractZendeskTicketDataFromParentMessage({
      slackMessage: new SlackMessage(parentBlocks),
    }),
    countryCode: channelCountryCode,
  });

  const getTicketDataWithFirstReplyData = (
    {
      extractReasonKey,
    }: {
      extractReasonKey: boolean;
    } = {
      extractReasonKey: false,
    },
  ): TicketCreate => {
    const replyBLocks: SectionBlock[] = result.messages?.[1].blocks as SectionBlock[];
    return {
      ...extractZendeskTicketDataFromParentMessage({
        slackMessage: new SlackMessage(parentBlocks),
      }),
      ...extractZendeskTicketDataFromFirstThreadReply({
        slackMessage: new SlackMessage(replyBLocks),
        extractReasonKey,
      }),
      countryCode: channelCountryCode,
    };
  };

  return {
    ...parentMessage,
    getTicketData,
    getTicketDataWithFirstReplyData,
    getAllTicketData: (): TicketCreate => ({
      ...getTicketData(),
      ...getTicketDataWithFirstReplyData(),
      subject: `Order Reference: ${getTicketData().orderReference} - ${
        getTicketDataWithFirstReplyData({ extractReasonKey: true }).reason
      }`,
    }),
    getFirstThreadReplyId: (): string => String(result.messages?.[1].ts),
  };
};

I am using getParentData to get the info from main thread before posting data to an API. I am thinking of reducing these calls by implementing a lru-cache?

Expected result:

The Slack app should work without hitting the rate limits by optimizing API calls.

Actual result:

The app is constantly hitting Slack API rate limits, causing interruptions in the app's functionality.

Dependencies:

"dependencies": {
  "@slack/bolt": "^3.11.0",
  "@slack/web-api": "^6.7.1",
  "axios": "^0.21.4",
  "axios-auth-refresh": "^3.3.1",
  "body-parser": "^1.20.0",
  "dotenv": "^16.0.1",
  "express": "^4.18.1",
  "express-validator": "^6.14.0",
  "form-data": "^4.0.0",
  "http-status-codes": "^2.2.0",
  "joi": "^17.6.0",
  "module-alias": "^2.2.2",
  "pino": "^7.11.0",
  "prom-client": "^12.0.0",
  "swagger-ui-express": "^4.3.0"
},
"devDependencies": {
  "@types/express": "^4.17.13",
  "@types/hapi__joi": "^17.1.8",
  "@types/jest": "^27.5.0",
  "@types/node": "^17.0.32",
  "@types/swagger-ui-express": "^4.1.3",
  "@typescript-eslint/eslint-plugin": "^5.23.0",
  "@typescript-eslint/parser": "^5.23.0",
  "axios-mock-adapter": "^1.20.0",
  "eslint": "^8.15.0",
  "eslint-config-airbnb-typescript": "^17.0.0",
  "eslint-config-prettier": "^8.5.0",
  "eslint-plugin-import": "^2.26.0",
  "eslint-plugin-jsx-a11y": "^6.5.1",
  "eslint-plugin-prettier": "^4.0.0",
  "husky": "^7.0.4",
  "install": "^0.13.0",
  "jest": "^28.1.0",
  "lint-staged": "^12.4.1",
  "nodemon": "^2.0.16",
  "prettier": "^2.6.2",
  "ts-jest": "^28.0.2",
  "ts-node": "^10.7.0",
  }

What else besides lru-caching can i introduce to atleast reduce the amount of slack API rate limits. API throttling, retries with exponensial backoff?

Thanks in advance, Erkand.

seratch commented 1 year ago

Hi @erkand-imeri, thanks for writing in!

One approach you may be interested would be the "smart" rate limiter in our Java SDK:

When using the async web client provided by the SDK, it automatically controls the traffic to Slack API servers under the hood. This enables developers' apps to maintain an optimal pace and avoid being rate-limited. The module manages a queue and traffic metrics to decide how long to wait before making the next API call.

Once your app receives rate-limited errors, the total duration to complete the same number of API calls would be significantly longer. Therefore, monitoring the situation and having pauses between API calls is the best way to mitigate the risk of errors.

Unfortunately, we don't have plans to add something similar to this Node SDK, at least in the short term (due to our bandwidth, priorities, and the current design of the Node SDK). So, please take a look at the Java SDK implementation to get inspiration for your code.

I hope this helps.

github-actions[bot] commented 1 year ago

👋 It looks like this issue has been open for 30 days with no activity. We'll mark this as stale for now, and wait 10 days for an update or for further comment before closing this issue out. If you think this issue needs to be prioritized, please comment to get the thread going again! Maintainers also review issues marked as stale on a regular basis and comment or adjust status if the issue needs to be reprioritized.

erkand-imeri commented 1 year ago

It looks like implementing lru-caching helped me a lot to avoid hitting the slack api rate limit.

CrazyGang97 commented 10 months ago

@erkand-imeri
hi Can you tell me how to use LRU to prevent rate limiting?

gitcommitshow commented 3 months ago

When using the async web client provided by the SDK, it automatically controls the traffic to Slack API servers under the hood.

Can you please provide the reference module file which does this? @seratch

From the behavior I observe in my implementation, it looks like that it doesn't anticipate the rate limits before making the api call, rather just reacts to the rate limited error by retrying the same api again. And in the retry, it fails as well.

So my understanding is that the developer will always need to implement their own RateLimiter.

Am I correct?

seratch commented 3 months ago

Yes, you're right. The Java SDK implements rate limiter to avoid getting 429 responses as much as possible. Checking the following resources may be helpful for you:

slackapi / bolt-js