search

tumblr / docs

Tumblr's public platform documentation.
Apache License 2.0
109 stars 27 forks source link

Random HTTP 429 rate limit response? #139

Open RagingLeviathan opened 1 month ago

RagingLeviathan commented 1 month ago

Hey all, I built a node.js app that responds to discord message triggers. All it does is call the /followers endpoint when invoked and save the response to a json file.

Been using it for months without issue, but today, for some reason, I keep running into a rate limit exceeded? I've checked the headers of the response received from Tumblr's API, and i don't seem to be hitting the limit? It should be about 50ish requests a minute as it iterates through my followers list.

Response headers: Object [AxiosHeaders] { server: 'nginx', date: 'Sat, 28 Sep 2024 18:14:38 GMT', 'content-type': 'application/json; charset=utf-8', 'content-length': '89', connection: 'keep-alive', 'x-rid': '64fd05ca71d0dc344741f924217e79d2', p3p: CP="Tumblr's privacy policy is available here: https://www.tumblr.com/policy/en/privacy", 'x-ratelimit-perday-limit': '5000', 'x-ratelimit-perday-remaining': '4106', 'x-ratelimit-perday-reset': '76341', 'x-ratelimit-perhour-limit': '1000', 'x-ratelimit-perhour-remaining': '610', 'x-ratelimit-perhour-reset': '753', 'strict-transport-security': 'max-age=31536000; preload', 'alt-svc': 'h3=":443"; ma=86400' }

calling ; https://api.tumblr.com/v2/blog/${blogName}/followers?limit=2000&offset=20&prev_offsets=0

Yep, I did set the followers limit ridiculously high! Any clue what could be up?

The strangest thing is i am seeing it retrieving blogs, but it seems to fail before the fetching completes.

Tried to contact support, but got a response from a bot. :')

RagingLeviathan commented 1 month ago

Speak of the devil, as soon as i post this, it finally completed. 🤣 Could this have been a transient issue on Tumblr's side maybe? Strange one, for sure. 🤔

RagingLeviathan commented 1 month ago

Although, hmm, maybe not. 🤔 This is happening again... with the first call I tried today? It's like there's an issue with trying to retrieve the last follower, has anyone got any ideas what could be up?

cyle commented 1 month ago

I didn't even know we had this, but it turns out we have a separate rate limit for the followers endpoint, ~60 requests per minute. You likely hit that! I'll add this to our documentation. Sorry about that!

cyle commented 1 month ago

Looks like you also uncovered a small bug, the x-ratelimit-perhour-limit is telling you about the global rate limit that applies to all of your consumer's API usage, and not the limit that applies to this specific endpoint. We can probably fix that up too, it would've made this a bit clearer, I hope.

RagingLeviathan commented 1 month ago

Oh, whoops! Thank you for sharing this, i appreciate it! :) This definitely helps, so cheers!

RagingLeviathan commented 1 month ago

Looks like you also uncovered a small bug, the x-ratelimit-perhour-limit is telling you about the global rate limit that applies to all of your consumer's API usage, and not the limit that applies to this specific endpoint. We can probably fix that up too, it would've made this a bit clearer, I hope.

Will the rate limit for this specific endpoint come back in the headers in future too, or at least be captured in the documentation?

cyle commented 1 month ago

at the very least, we'll add documentation about it, for sure.

i need to confer with the folks closer to the work on how these headers are meant to behave before i can say whether we should reflect the "global" or "local" rate limits there.