Deltares / ddlpy

API to Dutch Rijkswaterstaat archive (DDL, waterinfo.rws.nl) of monitoring water data
https://deltares.github.io/ddlpy/
GNU General Public License v3.0
20 stars 6 forks source link

add function for number of measurements per year/day #36

Closed veenstrajelmer closed 6 months ago

veenstrajelmer commented 6 months ago

Description

Follow up from https://github.com/openearth/ddlpy/issues/34. The option to retrieve the number of measurements per year from the webwaterservices is not exposed in ddlpy. Alternative example for hoekvhld large data gaps example code in https://github.com/Deltares-research/kenmerkendewaarden/issues/4

What I Did

import requests
import pandas as pd
import datetime as dt
import pytz

dtstart = dt.datetime.now()

# get number of waarnemingen
# TODO: takes long for long timeseries
url_ddl = 'https://waterwebservices.rijkswaterstaat.nl/ONLINEWAARNEMINGENSERVICES_DBO/OphalenAantalWaarnemingen'
request_ddl = {"AquoMetadataLijst" :
               [{"Compartiment":{"Code":"OW"},"Eenheid":{"Code":"cm"},
                 "Grootheid":{"Code":"WATHTE"},
                 "Hoedanigheid":{"Code":"NAP"},
                 "Groepering":{"Code":"NVT"},
                 }],
                "Groeperingsperiode" : "Dag",
               "LocatieLijst" : [{"X" :518882.333320247,"Y" :5760829.11729589,"Code":"EURPFM"}],
               # "Periode" : {"Begindatumtijd" : "1912-01-16T14:00:00.000+01:00","Einddatumtijd": "2024-01-16T16:00:00.000+01:00"}
               "Periode" : {"Begindatumtijd" : pytz.UTC.localize(dt.datetime(1900,1,1)).isoformat(timespec="milliseconds"),
                            "Einddatumtijd": pytz.UTC.localize(dt.datetime.now()).isoformat(timespec="milliseconds")}
               }
resp = requests.post(url_ddl, json=request_ddl)
if not resp.ok:
    raise Exception('%s for %s: %s'%(resp.reason, resp.url, str(resp.text)))
result = resp.json()
if not result['Succesvol']:
    raise Exception('query not succesful, Foutmelding: %s from %s'%(result['Foutmelding'],url_ddl))
for one in result['AantalWaarnemingenPerPeriodeLijst']:
    print(one['AquoMetadata']['Grootheid'])
    print(one['AquoMetadata']['Hoedanigheid'])
    print(one['AquoMetadata']['Groepering'])
    # print(one['AantalMetingenPerPeriodeLijst'])
    df = pd.json_normalize(one['AantalMetingenPerPeriodeLijst'])
    print(df)
print(f'retrieving took: {(dt.datetime.now()-dtstart).total_seconds():.2f} sec')

date_min = dt.datetime(df.iloc[0]["Groeperingsperiode.Jaarnummer"],
                       df.iloc[0]["Groeperingsperiode.Maandnummer"],
                       df.iloc[0]["Groeperingsperiode.Dag"])