projects.knmi.nl is taken offline

[jvansijl]

projects.knmi.nl seems to be down (permanently?) the description page is still on it seems: https://www.knmi.nl/kennis-en-datacentrum/achtergrond/data-ophalen-vanuit-een-script

[rubencalje]

Yes, it was offline yesterday (17-2-2021) as well. I do not think (hope) it will be down permanently, but we just have to wait....

[wcoppens]

Well it was announced it would get shutdown on 1 juli 2021 and an alternative was in development. You can still read the notification by looking at the cached pages by Google: http://webcache.googleusercontent.com/search?q=cache:zNXPpbtBA2wJ:projects.knmi.nl/klimatologie/daggegevens/index.cgi%3Fyear%3D1909%26month%3D8%26day%3D13+&cd=1&hl=en&ct=clnk&gl=nl

[jvansijl]

I read here that the KNMI data centre was superseded by the KNMI data platform on the 20th of july 2020: https://dataplatform.knmi.nl/open-data-info/index.html

KNMI now works with API keys, so slightly less user friendly and accessible than the CGI scripts I suppose....

[wcoppens]

@jvansijl Thats true, however the Data centre API you mention wasn't being used in this case.

I've had contact with them and they told me it's a downtime issue causing these problems. They also updated their pages today to inform there are ongoing issues. (see https://www.knmi.nl/kennis-en-datacentrum/achtergrond/data-ophalen-vanuit-een-script)

Hopefully they get it fixed soon.

[dbrakenhoff]

Thanks for the update! Good to know it's a technical issue on their end. I started looking into the KNMI data platform, but wasn't able to find an API endpoint for the data we usually access (i.e. historic timeseries per station for precipitation/evaporation).

Some of the products might still be interesting though, e.g. historic precipitation timeseries modified according to different future climate scenarios. Also gridded datasets for precipitation and evaporation based on interpolation between stations. These could interesting to check out someday...

[OnnoEbbens]

I think they do have something but I am still struggling.

What I found so far:

• dataset with meteo data. I think this is what we want. endpoint: https://api.dataplatform.knmi.nl/open-data/datasets/etmaalgegevensKNMIstations/versions/1/files dataset description: https://dataplatform.knmi.nl/catalog/datasets/index.html?x-dataset=etmaalgegevensKNMIstations&x-dataset-version=1

Unfortunately when I try to download using their api or via the website I get a netcdf file of 21kb. The structure of the netcdf file looks exactly as I would expect for the meteo station data. The only problem is, there is no data in it.

• dataset with evaporation data: endpoint: https://api.dataplatform.knmi.nl/open-data/datasets/EV24/versions/2/files dataset description: https://dataplatform.knmi.nl/catalog/datasets/index.html?x-dataset=EV24&x-dataset-version=2

I found an e-mail adress in the dataset description so I've send him an e-mail.

[rubencalje]

What we can do is download the data in an automated way from the zip-files of the KNMI.

For the meteological stations the url's can be found in: https://www.knmi.nl/nederland-nu/klimatologie/daggegevens

For the precipitation stations the url's can be found in: https://www.knmi.nl/nederland-nu/klimatologie/monv/reeksen

I have some code that reads the data from these locations, but we just have to make it work in hydropandas yet...

[OnnoEbbens]

fixed in the latest release (0.3.7)

[rubencalje]

The service is avaliable again, but the url's have changed. I committed new url's in commit fc7ef9d085ab6b4aee9b2774205e2b408021f13e. The format of the result has changes somewhat however, so the reads fail right now.

[OnnoEbbens]

fixed with the latest commit