There's Retry after exponential backoff delay already enabled in the topic so any rate limit should be retried correctly.
I'm also adding:
Dead Letter topic. So we don't lose any message that still may not be processed and ease monitoring and alerting
Random jitter inside the function so when the exponential backoff is elapsed we don't get all retries done at the same time.
Not sure if we have slack alert integrations in case we detect rate limit issues of topics increasing.
If this is not enough, in the future we could:
Send some kind of token summary of the content to avoid hitting the token limit. It would be naive to just grab a random chunk of the content because the offensive part could be in another place but there may be other options we could investigate
Fill the Increase Request Limit Form but we'd need to probe we really need that
dcadenas
commented
1 year ago
There's
Retry after exponential backoff delayalready enabled in the topic so any rate limit should be retried correctly.I'm also adding:
If this is not enough, in the future we could: