Error While Querying for Stations Using DwdObservationRequest

[nhelsherif]

Describe the bug Hi All

Seems that there is an error while trying to query for the Stations . My Code now fails when requesting the data with the following error

Traceback (most recent call last):
 .........
 .........
 .........
  File ""#####.py"", line 180, in get_stations_information_for_parameters
    result = request.all()
             ^^^^^^^^^^^^^
  File "/#####/.venv/lib/python3.12/site-packages/wetterdienst/core/timeseries/request.py", line 640, in all
    df = df.collect()
         ^^^^^^^^^^^^
  File "/#####/.venv/lib/python3.12/site-packages/polars/lazyframe/frame.py", line 1700, in collect
    return wrap_df(ldf.collect())
                   ^^^^^^^^^^^^^
polars.exceptions.ComputeError: conversion from `str` to `f64` failed in column 'longitude' for 524 out of 524 values: ["02     50", "80     47", … "54     48"]

To Reproduce I am using the following function to get the stations details

def get_stations_information_for_parameters(
        from_datetime, to_datetime, weather_parameters, resolution, observation_period
) -> StationsResult:
    """
    Function to get the stations information that will be used to get the weather data

    Args:
        from_datetime (datetime): Datetime from which we are getting data
        to_datetime (datetime): Datetime to which we are getting the data
        weather_parameters (list): List of Weather Parameters to request
        resolution (DwdObservationResolution): Data resolution parameter e.g. Hourly. For more details check Wetterdienst Python Lib
        observation_period (DwdObservationPeriod): Data Observation Period parameter e.g. recent. For more details check Wetterdienst Python Lib

    Returns:
       StationsResult: Object/data structure that contains the Stations information
    """
    request = DwdObservationRequest(
        parameter=weather_parameters,
        resolution=resolution,
        period=observation_period,
        start_date=from_datetime.strftime(REQUEST_DATE_FORMAT),
        end_date=to_datetime.strftime(REQUEST_DATE_FORMAT),
    )
    result = request.all()
    logging.info("Total number of Stations found: %s", result.df.shape[0])
    return result

Expected behavior Until Yesterday I was able to get the station data

Screenshots NA

Desktop (please complete the following information):

• OS: [ Mac-OS]
• Python-Version [3.12.1]
• Tested with Lib version 0.72.0 , 0.80.0 and 0.81.0

Additional context NA

[c0nb4]

We currently run into the same error.

Tested on

• docker
• Python 3.10.
• wettterdienst version 0.55.1

Our Error Code is:

` df[Columns.LONGITUDE.value] = df[Columns.LONGITUDE.value].astype(float)

ValueError: could not convert string to float: "44 52" `

[JBdama]

We are having the same problems.

Even some of the examples don't work anymore (discover() still works, but the rest not). I tried different python and wetterdienst versions (>0.72.0) on a fresh install.

The DwdMosmixRequest does not seem affected by this, only the DwdObservationRequest.

[safersephy]

Same problems here! Started yesterday with only the Minute 10 resolution giving this problem, now the daily resolution is also giving the same error

#collect temperatures   
stations = DwdObservationRequest(
parameter=Parameter.TEMPERATURE_AIR_MEAN_200,
#resolution=Resolution.MINUTE_10, #TODO wait till bugfix wetterdienst
resolution=Resolution.DAILY, 
start_date=datetime(2022, 8, 1),
end_date=datetime(2023, 9, 1),
)
# Filter on stationID closest to the garbage truck
result = stations.filter_by_station_id(station_id=("01303", ))

[LalonSander]

We have the same problem here and have written our own code now to retrieve the data we require.

The error seems to happen whe wetterdienst accesses the station list in this file: https://opendata.dwd.de/climate_environment/CDC/observations_germany/climate/daily/kl/recent/KL_Tageswerte_Beschreibung_Stationen.txt

The file was updated on April 25 at 9:20 (see update time here: https://opendata.dwd.de/climate_environment/CDC/observations_germany/climate/daily/kl/recent/ ) and probably something about the spacing between columns was changed. The file is tab/whitespace-separated.

[gutzbenj]

Dear @nhelsherif , Dear all,

thanks for reporting!

will be looking into this later this day and create a fix + release.

[gutzbenj]

The new release has a fix using new column specs for the parsing of the station lists. Apparently there was another issue for 5 minutes precipitation which has another station list at [1]. Counting the workarounds we already have for the DWD 😵‍💫

[1] https://opendata.dwd.de/climate_environment/CDC/observations_germany/climate//5_minutes/precipitation/historical/2022/5min_rr_Beschreibung_Stationen.txt

[ericmorelle]

Wow, I encounterd this issue yesterday and was happy to see it already being discussed. Not even 24h later and it is fixed!

Huge props to @gutzbenj and everyone involved!