YowanR
commented
2 years ago I've created an issue to test our checkpoint/resume feature and ensure that they are working correctly. In theory, even if we hit the backoff loop, the connector should be able to resume from where it left and not force the user to restart their long-running sync. cc @sherifnada
Tell us about the problem you're trying to solve
Twilio's API has a rate limit per request in a certain period of time. After a long-running sync process, it seems that enters into a backoff loop, and finally, when it reaches the sixth attempt the process fails and starts over again since the beginning. This happens when we try to synchronize large amounts of information, for example with the messages that we could have hundreds of millions.
Describe the solution you’d like
One type of handler that enables to improve the connector's sync, maybe to have a sleep time after making 6 backoff retries or have more than 6 retries to renew this quota limit and don't finish the whole process.