search

astoeckel / pydwdapi

Simple HTTP REST server for interpolated access to DWD german GDS station data
4 stars 2 forks source link

Time series forecast? #4

Open drmrbrewer opened 7 years ago

drmrbrewer commented 7 years ago

Not so much an issue, but more of a question...

This project looks really promising. I've been looking for a way of accessing the data from DWD without having to re-invent the wheel.

It seems that the json response from your API relates to the current weather at the specified lat, long. Is it possible to return a time-series forecast into the future, i.e. weather parameters for each hour over the next 48 hours or so?

Is this information even available from the raw DWD data? I did try to delve into the data at DWD's ftp server, but couldn't really understand the format of it.

astoeckel commented 7 years ago

Thank you for your question!

First of all, the DWD is currently restructuring the way in which they are providing data to the public. There is a new server http://opendata.dwd.de/weather/ which contains the data in a (possibly, I haven't looked into it) new format without registration. The GDS server used by pydwdapi will be obsolete by the end of 2017.

Second, you're right that this library does not extract any forecast data. This kind of data is definitively available from the DWD (albeit for a different set of locations). I don't extract it, since I was only interested in the current (and past) conditions. So, unfortunately, I guess that this library won't really help you in the current form. However, I'll gladly accept PRs implementing the extraction of the information from the new opendata server and to provide forecast information. For now I'm less interested in doing this myself, since I'm not located in Germany.

See https://www.dwd.de/DE/leistungen/opendata/help/inhalt_allgemein/opendata_content_de_en_xls.html?nn=495490 for a table containing all available open data sets and links to the corresponding documentation.

drmrbrewer commented 7 years ago

Thanks for the helpful response!

It seems to me that for time-series data the most appropriate dataset is this one. Would you agree? Within the mos/ subfolder there are files which each have data for all stations, while in the poi/ subfolder there are individual files for the various stations.

There is more information about this data here. What confuses me a little is that this appears to be a global dataset, and maybe there is another (maybe more detailed) dataset I'm missing that covers Germany only.

Anyway, from the above guide, it appears that a new KML-based file format may be introduced from next year, so it may be best if I don't put too much development effort in right now because it seems like things are in the process of changing.

astoeckel commented 7 years ago

The MOS/POI dataset you linked to was what I found as well, and yes, it doesn't offer a high spatial resolution for Germany. For a high spatial resolution you should probably use the raw data from the ICON forecast model, however, that data seems to be non-trivial to parse (though the GRIB2 format it uses seems to be standardised, so there surely is a Python library around).

drmrbrewer commented 7 years ago

Thanks again. I have in the past dabbled with trying to parse GRIB2 files, and nearly lost the will to live. So I think I'll stick with the mos and poi data and see where it gets me.