earthobservations / wetterdienst

Open weather data for humans.
https://wetterdienst.readthedocs.io/
MIT License
342 stars 52 forks source link

Stations list longitude and latitude not properly parsed #1283

Closed guidocioni closed 2 months ago

guidocioni commented 2 months ago

Describe the bug I don't know when this started happening, it was fine just few days ago and I haven't changed any package version (I think) :) Apparently when trying to get a station list I get this error

ComputeError: Conversion from `str` to `f64` failed in column 'longitude' for 1359 out of 2717 values: ["478     47.", "202     50.", … "215     50."]

Maybe something changed on the DWD server that is screwing up the space padding?

To Reproduce

from wetterdienst.provider.dwd.observation import DwdObservationRequest, DwdObservationDataset
from wetterdienst import Resolution, Settings

stations = DwdObservationRequest(
    parameter=DwdObservationDataset.CLIMATE_SUMMARY,
    resolution=Resolution.DAILY,
)
stations = stations.all().df
guidocioni commented 2 months ago

yep, something weird going on when parsing the data...

from wetterdienst.provider.dwd.observation import metaindex 

metaindex._create_meta_index_for_climate_observations(DwdObservationDataset.CLIMATE_SUMMARY,
                                                     Resolution.DAILY,
                                                     DwdObservationPeriod.HISTORICAL,
                                                     Settings(ts_shape="wide", ts_humanize=True, ts_si_units=False)).collect()

returns

station_id start_date end_date height latitude longitude name state
str str str str str str str str
"00001" null "1937" "370101 1986063… "4" "478 47." ".8413 8.849… "Baden-Württemb…
null null null null null null null null
"00003" null "1891" "910101 2011033… "2" "202 50." ".7827 6.094… "Nordrhein-West…
null null null null null null null null
"00011" null "1980" "800901 2024042… "6" "680 47." ".9736 8.520… ") …
null null null null null null null null
"00044" null "1969" "690101 2024042… null "44 52." ".9336 8.237… "Niedersachsen"
null null null null null null null null
"00052" null "1969" "690101 2001123… null "46 53." ".6623 10.199… "Schleswig-Hols…
null null null null null null null null
"00061" null "1975" "750701 1978083… "3" "339 48." ".8443 12.617… "Bayern"
null null null null null null null null
null null null null null null null null
"19631" null "1881" "810601 1969123… "2" "268 50." ".8083 10.229… "Thüringen"
null null null null null null null null
"19647" null "1951" "510101 2005103… "1" "178 49." ".4547 8.979… "Baden-Württemb…
null null null null null null null null
"19774" null "1971" "710819 1994022… "1" "174 51." ".4814 10.805… "Thüringen"
null null null null null null null null
"19781" null "1941" "410101 1953123… "3" "367 48." ".7429 11.423… "Bayern"
null null null null null null null null
"19794" null "1947" "470101 1954123… "3" "370 49." ".4415 11.852… "Bayern"
null null null null null null null null
"19911" null "1963" "631101 1986093… "2" "215 50." ".9625 10.066… "Thüringen"

investigating why....

guidocioni commented 2 months ago

Sorry, duplicate of https://github.com/earthobservations/wetterdienst/issues/1277