zorkian / go-datadog-api

A Go implementation of the Datadog API.
BSD 3-Clause "New" or "Revised" License
183 stars 156 forks source link

Adding logic to extract the Rate Limiting statistics #291

Closed CharlyF closed 4 years ago

CharlyF commented 4 years ago

In an effort to give maximum visibility over the communication between the client and the backend, we would like to introduce logic to extract the Rate Limiting stats from the header of calls made to rate limited endpoints. More details in the official API doc.

It exposes a thread safe public method to get the values. Additionally, we update the map associated with the Client in the method closest to the request to ensure that we don't miss calls.

CharlyF commented 4 years ago

Thank you for the review @gzussa.

Do we need to make this a cached/async thingy?

I believe clients should be as lightweight and stateless as possible, we need to keep the values in memory so that one can use them anytime, in my use case I submit them as metric to be alerted if I am about to reach the limit:

Image 2019-12-30 at 3 33 15 PM

I don't think we need a full cache though.

For the async, it is critical to be able to know when this problem arises, my approach was to 1. Get the most up to date values upon demand, 2. Not contribute to the Rate Limit when requesting the statistics (i.e. we don't have a rate limiting endpoint, so you need to query a rate limited endpoint to get them). Exactly as you said:

you can get latest values independently from your api calls.

With regards to the following,

Also, I fear we will forget to add new endpoints to the list of supported endpoints.

I added logic to parse the header of every call and look for the X-RATELIMIT-, you were right, this makes us more future proof.