Rijkswaterstaat / wm-ws-dl

wm-ws-dl documentation
https://rijkswaterstaatdata.nl/waterdata
11 stars 2 forks source link

difficult to get the latest time since multiple or no results are returned per location #15

Open veenstrajelmer opened 6 months ago

veenstrajelmer commented 6 months ago

Would be convenient if start/stoptimes of all stations can be retrieved instantaneously. In the below example the requested location list and the returned values do not have the same lenght, so they cannot be matched anymore. It would be great to have a location list and a result list that have the same lenght. Or include the first/last time in the location/aquometadata catalog.

import ddlpy
import requests

aquometadata_dict = {
    # "Eenheid": {"Code": location["Eenheid.Code"]},
    "Grootheid": {"Code": "WATHTE"},
    "Hoedanigheid": {"Code": "MSL"},
    # "Compartiment": {"Code": location["Compartiment.Code"]},
    # "Parameter": {"Code": location["Parameter.Code"]},
    "Groepering": {"Code": "NVT"},
    "MeetApparaat": {"Code": "127"},
}

# locatie_dict = 
locations_ddlpy = ddlpy.locations()
bool_grootheid = locations_ddlpy["Grootheid.Code"].isin(["WATHTE"])
bool_hoedanigheid = locations_ddlpy["Hoedanigheid.Code"].isin(["MSL"])
selected = locations_ddlpy.loc[bool_grootheid & bool_hoedanigheid]

print("collecting location dicts")
loc_list = []
for stat_code, location in selected.iterrows():
    locatie_dict = {
        "X": location["X"],
        "Y": location["Y"],
        "Code": location.get("Code", location.name),
    }
    loc_list.append(locatie_dict)
    if len(loc_list) == 100:
        break

print("request")
request = {"AquoPlusWaarnemingMetadataLijst":[{"AquoMetadata":aquometadata_dict}],
            "LocatieLijst":loc_list
            }

url = "https://waterwebservices.rijkswaterstaat.nl/ONLINEWAARNEMINGENSERVICES_DBO/OphalenLaatsteWaarnemingen"
resp = requests.post(url, json=request, timeout=5)
result = resp.json()
if not result['Succesvol']:
    raise Exception(result.get('Foutmelding', 'No error returned'))

df = ddlpy.ddlpy._combine_waarnemingenlijst(result, location)
df_simple = ddlpy.simplify_dataframe(df)
print(df_simple)
print('#locations:', len(loc_list))
print('#results:', len(df_simple))

Prints:

                          WaarnemingMetadata.StatuswaardeLijst  ... Meetwaarde.Waarde_Alfanumeriek
time                                                            ...                               
1999-08-31 23:50:00+01:00                           Definitief  ...                            -86
2024-04-03 18:40:00+01:00                      Ongecontroleerd  ...                            NaN
2024-04-03 18:30:00+01:00                      Ongecontroleerd  ...                            NaN
2019-12-29 23:50:00+01:00                      Ongecontroleerd  ...                            -26
2024-04-03 18:40:00+01:00                      Ongecontroleerd  ...                            NaN
2024-04-03 18:40:00+01:00                      Ongecontroleerd  ...                            NaN
2024-04-03 18:30:00+01:00                      Ongecontroleerd  ...                            NaN
2024-04-03 18:30:00+01:00                      Ongecontroleerd  ...                            NaN
2024-04-03 18:30:00+01:00                      Ongecontroleerd  ...                            NaN
2024-03-05 14:20:00+01:00                      Ongecontroleerd  ...                            NaN
2024-04-03 18:40:00+01:00                      Ongecontroleerd  ...                            NaN
2024-04-03 18:40:00+01:00                      Ongecontroleerd  ...                            NaN
2024-04-03 18:40:00+01:00                      Ongecontroleerd  ...                            NaN
2024-04-03 18:30:00+01:00                      Ongecontroleerd  ...                            NaN
2024-04-03 18:30:00+01:00                      Ongecontroleerd  ...                            NaN
2024-04-03 18:40:00+01:00                      Ongecontroleerd  ...                            NaN
[16 rows x 9 columns]

#locations: 38

#results: 16
TvLoon-RWS commented 4 months ago

Can be accounted for in Wadar WaterWebservices with -for example- an extra endpoint