Open Limess opened 3 years ago
On further investigation, it seems this is only on startup, prior to the server starting, due to INCIDENT_BOT_ID
not being set. We've now set this and it has helped alleviate this issue as it avoids that initial lookup.
We're seeing issues starting up services due to rate limiting by slack on the users listing calls.
The current retry behaviour is fairly aggressive and retries 10 times with an initial delay of 0.2 seconds.
Slack send a
Retry-After
header when rate-limiting and suggest their rate limiting is minutely, but that header is not accessible using theapi_call
method on theslackclient
library (at least in v1, I haven't checked v2).Ideally this would respect that header and retry after a minute or so.
Logs
On startup:
During this time it seems the server is not available, which with large backoffs also result in our healthcheck killing the starting service after some timeout.