cooperspencer / gickup

https://cooperspencer.github.io/gickup-documentation/
Apache License 2.0
962 stars 34 forks source link

github: GetIssues loops indefinitely if request fails #255

Open joshuasing opened 1 month ago

joshuasing commented 1 month ago

The github.GetIssues function contains a for loop to handle issue pagination: https://github.com/cooperspencer/gickup/blob/d29454e85636b3b0886510c962e226eb03e3219c/github/github.go#L388-L402

However, when the request fails, instead of exiting or handling the error, it is logged and immediately retried. The retry happens without delay, and happens indefinitely or until the request succeeds.

When a permanent error occurs when getting issues, such as 403 Resource not accessible by personal access token [], this will loop indefinitely, resulting in the GitHub API being spammed and causing the account to be rate limited.

Instead of indefinitely retrying, there should be an errCount or similar variable that makes the function exit after a certain number of retries. It would also be nice if request were retried after a delay, with jitter. Ideally, the function would also return the error, however that would be an API breaking change.

This error made it near impossible to use my GitHub account for several days, as it was constantly rate limited and impossible to tell what was causing the rate limit to be hit. I am thankful that GitHub did not terminate my account for the amount of API requests that were being made.

cooperspencer commented 1 month ago

I'll have a look at it when I have access to my laptop again.

cooperspencer commented 4 weeks ago

I finally have access to my laptop again. https://github.com/cooperspencer/gickup/pull/256/files

Gickup will attempt to retrieve the issues up to five times. If it still fails, it will return the list of issues it has managed to collect so far. I added a special case for a 403 error, as it doesn't make sense to retry when permissions are lacking. In this case, it will try once, throw an error, and then continue with the backup.