MeltanoLabs / tap-gitlab

Singer.io Tap for extracting data from Gitlab's API
GNU Affero General Public License v3.0
11 stars 29 forks source link

bug: 429 Client Error: Too Many Requests for path #94

Open kgpayne opened 1 year ago

kgpayne commented 1 year ago

During testing with the new standard tests framework, I am hitting 429 Client Error: Too Many Requests for path errors for the /projects/../pipelines/ API.

According to the Mozilla http code reference:

A Retry-After header might be included to this response indicating how long to wait before making a new request.

If this is available from the GitLab API, we should use this to wait the appropriate time (rather than using exponential backoff, as is default in the SDK).

EDIT: this appears to be supported in the backoff python library here 🚀

AlejandroUPC commented 6 months ago

Is this repo somewhat still maintained? I think the backoff solution looks fine or something as simple as a variable to add a delay between requests that can be set as a config parameter would also work

edgarrmondragon commented 6 months ago

Is this repo somewhat still maintained? I think the backoff solution looks fine or something as simple as a variable to add a delay between requests that can be set as a config parameter would also work

@AlejandroUPC would you like to submit a PR?

AlejandroUPC commented 5 months ago

yes expect one today, been quite busy sorry