ArtesiaWater / hydropandas

Module for loading observation data into custom DataFrames
https://hydropandas.readthedocs.io
MIT License
52 stars 11 forks source link

add waterinfo api #93

Open OnnoEbbens opened 1 year ago

OnnoEbbens commented 1 year ago

There is already code to read a waterinfo csv file but no code yet to use an API to get waterinfo data. This package (https://github.com/openearth/ddlpy) uses the API so maybe we can use this package.

martinvonk commented 6 months ago

Some extra info from RWS: https://rijkswaterstaatdata.nl/waterdata/#hf1666c4e-b766-49e0-af37-c1b95f8f56e1

https://rijkswaterstaat.github.io/wm-ws-dl/#introduction

martinvonk commented 6 months ago

@rubencalje also wrote some code in art_tools which can be used. I think it is faster than ddlpy because Ruben chunks the data in bigger parts.

martinvonk commented 6 months ago

Maybe the Matroos api could work as well. I can download some files but they are empty ... From this example: https://github.com/openearth/sealevel/blob/62c8d6254ad42b491dd7777c3f53f60f190e4796/data/rws/matroos/download_matroos.ipynb

import yarl
import pathlib

stations = [
    "schoonhoven",
]

url = yarl.URL('http://noos.matroos.rws.nl/direct/get_series.php')

station = stations[0]

defaults = {
    "tstart": 201701010000,
    "tstop": 201901010100,
    "format": "text"
}
params_list = [
    {
        "source": "observed",
        "unit": "waterlevel",
        "name": "waterlevel_observed"
    },
]
for station in stations:
    for params in params_list:
        download_url = url.update_query(defaults).update_query(params).update_query({"loc": station})
        filepath = "{}_{}.txt".format(station, params["name"])
        !wget -c -O "$filepath" "$download_url"

print(pathlib.Path("schoonhoven_waterlevel_observed.txt").read_text())
martinvonk commented 6 months ago

Another repository that obtains data from waterinfo: https://gitlab.com/rwsdatalab/public/codebase/tools/rws-waterinfo