jdemaeyer / brightsky

JSON API for DWD's open weather data.
https://brightsky.dev/
MIT License
287 stars 18 forks source link

Getting many 429 errors lately when calling the API #162

Open BeyondVertical opened 6 months ago

BeyondVertical commented 6 months ago

Hello, I am getting many 429 errors lately - I did not change something significantly on my end. Has something changed on the API? Is there a rate limit now, maybe? What would that be, so that I can adjust accordingly. Thanks for your help!

tawissus commented 6 months ago

Hey, I have problems too. Both in my query and by calling the appropriate URL.

I have just received the following error: "Too Many Requests"

Some calls also work.

jdemaeyer commented 6 months ago

Hi guys, thanks for the report! I can confirm that I got a bunch of spurious 429s in manual testing.

There is indeed a rate limit of 10 requests per second (with a burst size of 50), but given that you haven't changed anything on your side I doubt you're hitting that. My first guess was that we're reaching the maximum capacity for our current setup, but upgrading the server hasn't fixed the situation.

My next guess is that either I screwed up configuring traefik's rate limit middleware or that that middleware has a bug. I have temporarily completely disabled rate limiting - can you confirm that you're no longer getting 429s?

tawissus commented 6 months ago

Hey, thanks for the quick response. I have now started my query a few times in a row and it has always worked. I can't say anything about the 429.

BeyondVertical commented 6 months ago

Hey @jdemaeyer, thanks for disabling the rate limit. I can confirm, that the requests are without errors since about half an hour. I am definitely not hitting the mentioned limit. I was testing with delays in the script yesterday (up to delaying a single request for a second) and that did also not improve anything. So I guess the problem could be in those regions you mentioned. Thanks for fixing it for the moment and looking into it.

jdemaeyer commented 6 months ago

Thanks a lot for the fast responses! I'll be playing around with the rate limit configuration for a bit so you might run into a few errors in the next hours