Closed amotl closed 1 year ago
@amotl Thank you for the report! I will have a look into why this is happening.
The summarize feature was done by @gutzbenj, looks really nice! I could not reproduce the wrong results locally on MacOS (Python 3.9/3.10). Just a guess but might be related to the old date in the test(1934-1965)?
Hitting this again. Do you have any idea, @gutzbenj?
Python 3.10, macOS. https://github.com/earthobservations/wetterdienst/actions/runs/3535301195/jobs/5933174921#step:7:744
Python 3.9, macOS. https://github.com/earthobservations/wetterdienst/actions/runs/3535529913/jobs/5933903562#step:7:734
Python 3.8, macOS. https://github.com/earthobservations/wetterdienst/actions/runs/3535529913/jobs/5934173889#step:7:734
It's a bummer that the test currently first compares values and afterwards the station id... otherwise we would know a certain station wasn't available for whatever reason.
Metadata still shows the same:
01048 19340101 20221122 228 51.1278 13.7543 Dresden-Klotzsche Sachsen
01050 19490101 20221122 112 -->51.0221 13.8470<-- Dresden-Hosterwitz Sachsen
01051 19360101 20221122 119 51.0249 13.7751 Dresden-Strehlen Sachsen
Also location is still the same:
stations = DwdObservationRequest(
parameter=Parameter.TEMPERATURE_AIR_MEAN_200,
resolution=DwdObservationResolution.DAILY,
start_date=datetime(1934, 1, 1),
end_date=datetime(1965, 12, 31),
)
result = stations.summarize(latitude=51.0221, longitude=13.8470) <--
Found it:
Apparently
05282 19170101 19740630 246 51.1197 13.6744 Wahnsdorf bei Dresden Sachsen
has temperature_air_mean_200
-0.2 °C -> 272.95 K on 1934-01-01 -7.0 °C -> 266.15 K on 1940-01-01 -4.4 °C -> 268.75 K on 1950-01-01
5282;19340101;-999;-999;-999; 5; 0.3; 8; 0.000; 3; 8.0; 6.0; 995.40; -0.2; 99.00; 0.1; -0.4;-999;eor
...
5282;19400101;-999;-999;-999; 5; 0.3; 7; 1.800; 13; 7.8; 3.2; 988.70; -7.0; 87.00; -3.1; -7.7;-999;eor
...
5282;19500101; 5;-999; 3.2; 5; 0.0; 0; 1.600; 0; 6.4; 3.6; 1000.30; -4.4; 79.00; -1.7; -5.0; -5.8;eor
Do you have a clue which of the options might have caused it?
Maybe add some log messages, which could give more hints about the root cause? On GHA, the hiccup happens quite often, so it shouldn't be hard to get corresponding traces.
Thanks for looking at the data.
Do you have a clue which of the options might have caused it?
I am looking at the computing environment.
The anomaly apparently is only present on macOS, with Python 3.8-3.10. While all tests succeeded on Linux, and macOS with Python 3.11, we can't get past the other Python versions on macOS.
Maybe it's enough to tag the test case as a @cflake
? Otherwise, I'd say let's just skip the test on macOS for now. And then see, what it will report on Windows ;].
So this to me looks like it's related to the cache, if it is platform specific...
Ok, let's purge them.
https://github.com/earthobservations/wetterdienst/actions/caches
With 903066c1c, I've disabled the test case on GHA/macOS for now. It's really a weird issue, and may want to be followed up upon.
This has been fixed with https://github.com/earthobservations/wetterdienst/commit/725b94591e6a3c2c0d4fc70b7a7da9c4c7c32fea#r92707270. Thanks, @gutzbenj!
Dear @neumann-nico and @gutzbenj,
I don't know much about the interpolation subsystem, I only just spotted some anomalies and wanted to share them with you.
With kind regards, Andreas.
Observations
Two CI runs failed at two spots.
Example
Thoughts
Is it a fluke, and will resolve when re-run? Or did some underlying data change? I will appreciate your thoughts on this, as I currently feel a bit in charge to take care about keeping a stable test suite, for a while ;].
Evaluation
Apparently, the issue is only on macOS, which is failing on lower Python versions. Only Python 3.11 works.
[^2]: Python3.10, macOS: https://github.com/earthobservations/wetterdienst/actions/runs/3516930241/jobs/5895858497#step:7:741