earthobservations / wetterdienst

Open weather data for humans.
https://wetterdienst.readthedocs.io/
MIT License
349 stars 54 forks source link

ValueError: only one file expected at latest weather stations #1265

Closed yannik207 closed 4 months ago

yannik207 commented 5 months ago

Describe the bug

I am currently trying to scrape all the weather stations that have the latest information of the weather data. So what I don't want are weather stations with only historical information that got shutdown over the time. What I can is: sending the request and get the stations information What I can't is: retrieving the weather data for all the stations. I can retrieve the data for historical weather data but at that moment I am trying to include a weather stations that got information from the last days it breaks with the error:

ValueError: only one file expected

To Reproduce Steps to reproduce the behavior:

from datetime import datetime
from wetterdienst import Settings
from wetterdienst.provider.dwd.observation import DwdObservationRequest, DwdObservationResolution, DwdObservationParameter

settings=Settings(cache_disable=False)
print(settings)
year = 2024
request = DwdObservationRequest(
    parameter=[DwdObservationParameter.DAILY.TEMPERATURE_AIR_MEAN_200],
    resolution="daily",
    start_date="1990-01-01",
    end_date="2024-01-01"
    )

df_temp = request.filter_by_station_id(station_id=["03404", "03101", "01246"])
print(df_temp)
for result in df_temp.values.query():
    print(result.df.tail(5))

Which steps had led to the error? which lines of code had you run?

The weather station with the ID 01246 let the code breaks. If you remove the 01246 the code works perfectly.

I've also tried to disable the cache etc.

Desktop (please complete the following information):

burrscurr commented 4 months ago

I experienced a similar (the same?) issue. For me, disabling the cache (Settings(cache_disable=True)) did work. I think DWD recently updated their historical data to also include 2023, so maybe that's how the cache can become invalid, since it is still containing the old -2022 filename variants that are not present on the server anymore.

gutzbenj commented 4 months ago

That's a good hint! On 2024-04-03 the historical files were updated (at least for daily climate summary) and the cache is probably invalid for a bit of an hour or so (I don't currently remember the cache ttl we set for the daily historical data index).

gutzbenj commented 4 months ago

Can this be closed or does the issue still exist?

yannik207 commented 4 months ago

Yes, can be closed! Thanks for explanation.