search

xdevplatform / open-evolution

Open evolution proposals for the Twitter API
https://twittercommunity.com/t/introducing-twitter-open-evolution/158770
41 stars 15 forks source link

Feature request: Get endpoint usage and rate limits #20

Open myConsciousness opened 2 years ago

myConsciousness commented 2 years ago

Hi amazing developers,

Is there a way to programmatically get rate limits for Twitter API v2.0 endpoints?

If not, I feel that it would be more convenient for developers to be able to programmatically retrieve metadata such as usage and rate limits for each endpoint.

Thank you!

albattran commented 2 years ago

Yes, there is the x-rate-limit-remaining HTTP header which can be used to get the rate limits. You should be using this header to know how many rate limits left. However, that header alone is not enough because it cannot always be used for few reasons:

  1. It's sometimes missing, it's rare but does happen leaving the caller confused of what the remaining limit is
  2. Service restarts, after rebooting, upgrading, or horizontal scaling So as you can see, you will need a backup method to fetch the rate limits.

Some of V2 APIs do return data in the v1.1 GET application/rate_limit_status but my understanding is that the endpoint will be retired, so eventually we would need to have a replacement for it in v2. Short of that, you would need to build your own infrastructure to keep tabs (knowing when to call, persisting across server restarts and scaling), and use the actual header in between calls on same server.

myConsciousness commented 2 years ago

Hi @albattran ,

Thanks for your sharing!

I'm very curious... I understand that the header may not include the limit information, do you know exactly how often this happens? Does this mean that only some of the limit information fields are missing or all of them?

If this is a negligible glitch I would like to use this header information. If we can get the limit information from the header, then API calls will be minimal.

albattran commented 2 years ago

It's a tiny tiny percentage when it goes missing, so yes using the header should always be the preferred method (we have logic that assumes the rate-limit-remaining > 0 when the header is missing, at the risk of getting an error) But we still need to use "GET application/rate_limit_status" on startups due to clustering. Honestly, neither solution is sufficient by itself in a cluster environment, both go hand-in-hand