search

DataDog / datadog-api-client-python

Python client for the Datadog API
https://pypi.org/project/datadog-api-client/
Apache License 2.0
114 stars 41 forks source link

list_ci_app_pipeline_events_with_pagination should not bubble up 429s #1532

Closed tsmsbit closed 1 year ago

tsmsbit commented 1 year ago

Is your feature request related to a problem? Please describe. The problem is that when I was using the list_ci_app_pipeline_events_with_pagination method, I got a 429 due to rate limiting. My understanding was that this method would handle pagination for me, including retrying 429s.

Describe the solution you'd like I would like list_ci_app_pipeline_events_with_pagination to retry 429s for me, by waiting for the reset period that was indicated in the response headers of the 429 response and then retrying. Ideally there would also be a way for the user to know how much time was spent waiting due to rate limiting.

Describe alternatives you've considered I could switch to using list_ci_app_pipeline_events instead of list_ci_app_pipeline_events_with_pagination and handle the 429s myself. I will also follow up with Datadog to get the limits increased to a threshold that will suffice for my needs, but that wouldn't solve the underlying issue.

therve commented 1 year ago

Hi,

We're working on retry support in https://github.com/DataDog/datadog-api-client-python/tree/add-retry. We should be able to combine this and pagination to what you describe. Thanks for your patience.

PatrickJordanCongenica commented 1 year ago

Adding support behind the request as I'm experiencing this with the list_logs_with_pagination method too.

therve commented 1 year ago

Hi, We just merged the PR adding retry support: https://github.com/DataDog/datadog-api-client-python#enable-retry, it will be in the next release. If you get a chance to try it let us know, thanks.