Honor requested time interval when acquiring historical data of the 10-minutes resolution

[amotl]

Observation

@wetterfrosch reported at https://github.com/earthobservations/wetterdienst/issues/250#issuecomment-734316263:

When I query e.g. the past year from historical (2019-01/2019-12), which is within the 10-year time range of a single file, all historical files are loaded: Currently I count up to three, beginning in 1992. This doesn't seem efficient. Are all of these values loaded into the RAM?

I want to add that he was trying to mangle data from all stations, without any filtering on the station identifiers.

Details

We see the files representing historical data within the 10-minutes resolution [1] to be bundled into different files for each observation site, each one holding 10 years worth of data. I haven't been exactly aware of that detail, so thanks!

10minutenwerte_TU_00003_19930428_19991231_hist.zip 09-Apr-2020 09:13             2948612
10minutenwerte_TU_00003_20000101_20091231_hist.zip 09-Apr-2020 09:13             4545156
10minutenwerte_TU_00003_20100101_20110331_hist.zip 09-Apr-2020 09:13              657685

Please note this is different with other data products like hourly resolution [2], where there is just one file per observation site.

stundenwerte_TU_00003_19500401_20110331_hist.zip   08-Sep-2020 14:11             2861205
stundenwerte_TU_00044_20070401_20191231_hist.zip   08-Sep-2020 14:11              600029
stundenwerte_TU_00052_19760101_19880101_hist.zip   08-Sep-2020 14:11              551265

[1] https://opendata.dwd.de/climate_environment/CDC/observations_germany/climate/10_minutes/air_temperature/historical/ [2] https://opendata.dwd.de/climate_environment/CDC/observations_germany/climate/hourly/air_temperature/historical/

Expected behavior

Only acquire historical data from the appropriate 10-year interval in order to improve efficiency and save resources.

[gutzbenj]

For this to work, we'll have to

• parse the date from the filename to apply a filter around the year
• extract the datetime string e.g. _1993042819991231 and add this to the key used for the intermediate storage

[gutzbenj]

See #267