Deltares / ddlpy

API to Dutch Rijkswaterstaat archive (DDL, waterinfo.rws.nl) of monitoring water data
https://deltares.github.io/ddlpy/
GNU General Public License v3.0
19 stars 6 forks source link

consider simplifying try/except for Tijdstip Keyerror #102

Closed veenstrajelmer closed 4 months ago

veenstrajelmer commented 4 months ago

Description

Is there data without Tijdstip column? Either way, the try/except KeyError could be simplified by just throwing a KeyError

veenstrajelmer commented 4 months ago

This can be tested with this code:

import ddlpy

locations = ddlpy.locations()
bool_hoedanigheid = locations['Hoedanigheid.Code'].isin(['NAP'])
bool_stations = locations.index.isin(['HOEKVHLD', 'IJMDBTHVN','SCHEVNGN'])
bool_grootheid = locations['Grootheid.Code'].isin(['WATHTE'])
bool_groepering = locations['Groepering.Code'].isin(['NVT'])
selected = locations.loc[bool_grootheid & bool_hoedanigheid & bool_groepering & bool_stations]

start_date = "2019-01-01"
end_date = "2019-03-01"
# pass a single row of the locations dataframe to the measurements function to get the measurements for that location
measurements = ddlpy.measurements(selected.iloc[0], start_date, end_date)

And by updating that try/except:

    try:
        df["time"] = pd.to_datetime(df["Tijdstip"])
        df = df.set_index("time")
    except KeyError:
        logger.exception(
            "Cannot add time variable time because variable Tijdstip is not found"
        )

To something that will raise a KeyError:

    try:
        df["time"] = pd.to_datetime(df["Tijdstip_INVALIDKEYTOGETERROR"])
        df = df.set_index("time")
    except KeyError:
        logger.exception(
            "Cannot add time variable time because variable Tijdstip is not found"
        )

This will still return a correct dataframe, but with a different index. The Tijdstip column is still dropped, but this can be prevented with clean_df=False.

Therefore, in case if there is data without the Tijdstip column, this is a valuable feature to still let the process continue. It does not make a lot of sense to still log it as an exception, but this is also no huge issue. Therefore, this issue will be closed.