earthobservations / wetterdienst

Open weather data for humans.
https://wetterdienst.readthedocs.io/
MIT License
349 stars 54 forks source link

Quirk with power horse acquisition for data across historical+recent data sets #372

Closed amotl closed 3 years ago

amotl commented 3 years ago

Hi Benjamin,

I am currently giving cli.py a refresh and wanted to report about it.

This command line invocation listed in the examples section of wetterdienst --help croaked:

# The real power horse: Acquire data across historical+recent data sets
wetterdienst dwd observations values --station=1048,4411 --parameter=kl --resolution=daily --period=historical,recent --date=1969-01-01/2020-06-11
ValueError: 'recent,historical' is not a valid DwdObservationPeriod ```python Traceback (most recent call last): File "/Users/amo/dev/earthobservations/wetterdienst/wetterdienst/util/enumeration.py", line 40, in parse_enumeration_from_template enum_parsed = intermediate[enum_name] File "/usr/local/opt/python@3.8/Frameworks/Python.framework/Versions/3.8/lib/python3.8/enum.py", line 387, in __getitem__ return cls._member_map_[name] KeyError: 'RECENT,HISTORICAL' During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/Users/amo/dev/earthobservations/wetterdienst/wetterdienst/util/enumeration.py", line 43, in parse_enumeration_from_template enum_parsed = intermediate(enum_) File "/usr/local/opt/python@3.8/Frameworks/Python.framework/Versions/3.8/lib/python3.8/enum.py", line 339, in __call__ return cls.__new__(cls, value) File "/usr/local/opt/python@3.8/Frameworks/Python.framework/Versions/3.8/lib/python3.8/enum.py", line 662, in __new__ raise ve_exc ValueError: 'recent,historical' is not a valid DwdObservationPeriod During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/Users/amo/dev/earthobservations/wetterdienst/.venv/bin/wetterdienst", line 5, in run() File "/Users/amo/dev/earthobservations/wetterdienst/wetterdienst/cli.py", line 218, in run stations = DwdObservationRequest( File "/Users/amo/dev/earthobservations/wetterdienst/wetterdienst/dwd/observations/api.py", line 504, in __init__ super().__init__( File "/Users/amo/dev/earthobservations/wetterdienst/wetterdienst/core/scalar/request.py", line 182, in __init__ self.period = self._parse_period(period) File "/Users/amo/dev/earthobservations/wetterdienst/wetterdienst/core/scalar/request.py", line 126, in _parse_period pd.Series(period) File "/Users/amo/dev/earthobservations/wetterdienst/.venv/lib/python3.8/site-packages/pandas/core/series.py", line 4213, in apply mapped = lib.map_infer(values, f, convert=convert_dtype) File "pandas/_libs/lib.pyx", line 2403, in pandas._libs.lib.map_infer File "/Users/amo/dev/earthobservations/wetterdienst/.venv/lib/python3.8/site-packages/pandas/core/series.py", line 4198, in f return func(x, *args, **kwds) File "/Users/amo/dev/earthobservations/wetterdienst/wetterdienst/util/enumeration.py", line 45, in parse_enumeration_from_template raise InvalidEnumeration( wetterdienst.exceptions.InvalidEnumeration: recent,historical could not be parsed from DwdObservationPeriod. ```

However, everything works perfectly well when just omitting the --period parameter, like

wetterdienst dwd observations values --station=1048,4411 --parameter=kl --resolution=daily --date=1969-01-01/2020-06-11

Currently, I am very happy about this and only wanted to ask how we should proceed on this regarding rephrasing the example section.

With kind regards, Andreas.

gutzbenj commented 3 years ago

Probably we should just omit --period for the cli as most services are expected to have no differentiation of periods. Only for the DWD we can expect an overhead in the case that the request is overlapping a new years change.

amotl commented 3 years ago

All right, let's do it like that. I believe it is great that Wetterdienst already sorts this out under the hood so it frees the user from the obligation to think about this parameter. Great work from your side, I appreciate it!