search

launchdarkly-labs / project-migrator-script

Migrate Project Information from one account to another
3 stars 7 forks source link

Segment fails to migrate because of ratelimiting and too much async #16

Open sgandon opened 7 months ago

sgandon commented 7 months ago

The migrate script is making a lot of async calls (here and here). This means that all segment creation calls are almost launched at the same time. The rate limit if quickly reached and many calls wait for the ratelimit reset time together. And try again to call the endpoint altogether and eventually fails because of Too Many Requests.

cgreenld commented 6 months ago

Hey @sgandon appreciate your thoughts here - LaunchDarkly does not adjust the rate limits on our application site, so we are required to manage those async calls vs getting a limit moved. I understand this adds to the time needed to migrate with this script.

Open to suggestions on an optimization on the flow to possible avoid this, but there is a not a way to raise the limits

sgandon commented 6 months ago

I will suggest a PR to retry indefinitely and not break after the second retry. This eventually get things done.