Missing consideration of change of time stamp from MEZ to UTC of 1-minute / 10-minutes DWD data

earthobservations / wetterdienst

Open weather data for humans.

https://wetterdienst.readthedocs.io/

MIT License

347 stars 54 forks source link

Missing consideration of change of time stamp from MEZ to UTC of 1-minute / 10-minutes DWD data #541

Closed GeyerJoachim closed 2 years ago

GeyerJoachim commented 2 years ago

Describe the bug There is a change of the time zone in the time step of the »historical 10-minute station observations« product of the DWD, which is not considered by Wetterdienst. Citing from its dataset description (en, de):

The time stamp before the year 2000 is given in MEZ, the time stamp after the year 2000 is given in UTC. [en] Der Zeitstempel ist vor dem Jahr 2000 in MEZ gegeben, ab dem Jahr 2000 in UTC. [de]

amotl commented 2 years ago

Dear Joachim,

thank you very much for sharing this valuable bit of information with us. That definitively slipped through, we will look into how to compensate this detail. The best option would be to shift all timestamps before year 2000 to UTC, right?

With kind regards, Andreas.

GeyerJoachim commented 2 years ago

Dear Andreas,

yes, I just subtracted one hour from the time stamps before 2000. Thanks for your great work on Wetterdienst!

Joachim

gutzbenj commented 2 years ago

Dear @GeyerJoachim ,

after your issue I have contacted the DWD and they confirmed on your concerns. They have also told me that in the future there will be an update to the datasets that would include the missing hour - however there was no mentioning of any specific date.

A PR at #543 was set up that will fix this problem. Have you checked on the minutewise precipitation data and if this is also effected by the shifted timestamps?

gutzbenj commented 2 years ago

Checked it and 1 minute data is also shifted so will apply the same measures to that data. Attached is a PDF that visualizes this question.

wetterdienst – 10minute vs 1minute.ipynb.pdf

mhabedank commented 9 months ago

Hi folks, so I stumbled upon this problem in the data provided by Wetterdienst. The gap of an hour is still there. But at least for the 10 minute data, DWD added the missing data. Sadly they did this in a way that the timestamp overlap.