search

jdemaeyer / brightsky

JSON API for DWD's open weather data.
https://brightsky.dev/
MIT License
287 stars 18 forks source link

API responses very slow since about 3 pm CEST today #152

Closed BeyondVertical closed 1 year ago

BeyondVertical commented 1 year ago

Hey there, I am using your API and am running into timeouts since about 3 pm today. Your Grafana dashboards shows this too. Can you have a look at the server, when you have the time? Thanks for this great API and your work!

tawissus commented 1 year ago

True. For me, the last time access worked was around 16:00.

BeyondVertical commented 1 year ago

Yeah. Access works again now - shortly after midnight there were no timeouts. Thanks for fixing! :)

jdemaeyer commented 1 year ago

Hi guys, thanks for the report!

As you said, response times stabilized shortly after midnight. That's when most users changed their queries from date=2023-08-20 to date=2023-08-21 -- however, it doesn't seem that there were any changes in the underlying data in the middle of Sunday that would massively impact database query time like this. So I guess it was a hiccup that got reset by some daily maintenance task or rotation.

I'm swamped with work right now so can't investigate deeply, but I'll keep a closer look on the server throughout the day (my monitoring tools actually did pick up the outage yesterday, I just wasn't reachable :see_no_evil:)

smilingchemist commented 11 months ago

The problem seems to be back. Last record is from yesterday, 14:30 UTC...

BeyondVertical commented 11 months ago

The problem seems to be back. Last record is from yesterday, 14:30 UTC...

Yeah. Was gonna say this too. But it is a little different than last time. This time the response time seems fine, but the data is missing.

jdemaeyer commented 11 months ago

Thanks for reporting! New data is currently being parsed and should be available in the next 30 minutes.

This issue originated from a prolonged outage of dwd.de (not the open data server but their official website, from which we load a station list as a prerequisite for parsing) that we should build resilience against. I'll open a new issue in a bit

jdemaeyer commented 11 months ago

https://github.com/jdemaeyer/brightsky/issues/154