Open soxofaan opened 1 month ago
Re-enabling them on a local setup (python 3.11) fails due to usage of removed xarray.open_rasterio
functionality:
FAILED tests/test_integration.py::test_atmospheric_correction_inputsarecorrect[1.1.0] - AttributeError: module 'xarray' has no attribute 'open_rasterio' FAILED tests/test_integration.py::test_atmospheric_correction_defaultbehavior[1.1.0] - AttributeError: module 'xarray' has no attribute 'open_rasterio'
This would still work on jenkins, but that is basically a side-effect because we're stuck there on python 3.8 (which reaches EOL in a couple of months), which does not support newer xarray versions
Running them locally in an old python 3.9 venv fails like this for me:
________ test_atmospheric_correction_inputsarecorrect[1.1.0] ____________
...
compare_xarrays(result.loc[date][-5],vzaref[0].transpose("x","y"))
> compare_xarrays(result.loc[date][-4],raaref[0].transpose("x","y"))
tests/test_integration.py:1803:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
tests/test_integration.py:1760: in compare_xarrays
np.testing.assert_allclose(
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
args = (<function assert_allclose.<locals>.compare at 0x78708aa35dc0>, array([[13300., 13300., 13300., ..., 13400., 13400., 1... [13400., 13400., 13400., ..., 13300., 13300., 13300.],
[13400., 13400., 13400., ..., 13300., 13300., 13300.]]))
kwds = {'equal_nan': True, 'err_msg': '', 'header': 'Not equal to tolerance rtol=0, atol=1e-06', 'verbose': True}
@wraps(func)
def inner(*args, **kwds):
with self._recreate_cm():
> return func(*args, **kwds)
E AssertionError:
E Not equal to tolerance rtol=0, atol=1e-06
E
E Mismatched elements: 300000 / 400000 (75%)
E Max absolute difference: 100.
E Max relative difference: 0.0075188
E x: array([[13300., 13300., 13300., ..., 13400., 13400., 13400.],
E [13300., 13300., 13300., ..., 13400., 13400., 13400.],
E [13300., 13300., 13300., ..., 13400., 13400., 13400.],...
E y: array([[13400., 13400., 13400., ..., 13300., 13300., 13300.],
E [13400., 13400., 13400., ..., 13300., 13300., 13300.],
E [13400., 13400., 13400., ..., 13300., 13300., 13300.],...
../../../.asdf/installs/python/3.9.15/lib/python3.9/contextlib.py:79: AssertionError
_______ test_atmospheric_correction_defaultbehavior[1.1.0] _________
...
> compare_xarrays(result.loc[date,"B02"],b2ref[0].transpose("x","y"))
tests/test_integration.py:1835:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
tests/test_integration.py:1760: in compare_xarrays
np.testing.assert_allclose(
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
args = (<function assert_allclose.<locals>.compare at 0x78708a7b4700>, array([[484., 373., 340., ..., 374., 395., 381.],
...54., 659., 754.],
[278., 271., 269., ..., 602., 544., 765.],
[269., 262., 284., ..., 700., 728., 674.]]))
kwds = {'equal_nan': True, 'err_msg': '', 'header': 'Not equal to tolerance rtol=0, atol=1e-06', 'verbose': True}
@wraps(func)
def inner(*args, **kwds):
with self._recreate_cm():
> return func(*args, **kwds)
E AssertionError:
E Not equal to tolerance rtol=0, atol=1e-06
E
E x and y nan location mismatch:
E x: array([[484., 373., 340., ..., 374., 395., 381.],
E [339., 367., 371., ..., 401., 394., 374.],
E [478., 465., 414., ..., 425., 399., 395.],...
E y: array([[381., 395., 374., ..., 340., 373., 484.],
E [374., 394., 401., ..., 371., 367., 339.],
E [395., 399., 425., ..., 414., 465., 478.],...
../../../.asdf/installs/python/3.9.15/lib/python3.9/contextlib.py:79: AssertionError
I already tried to modernize/fix the tests, assuming it would just take half an hour (based on #20 experience), but it turns out quite a bit more debug work. Giving up for now and parking it in this ticket
while working on #20 I noticed these two test are marked as skipped since dec 2021:
https://github.com/Open-EO/openeo-geopyspark-integrationtests/blob/4ced825b3622d31da9f6da3b7b7462e56dbfc19f/tests/test_integration.py#L1769-L1838
We should re-enable them, or remove them. Keeping them like this in the test suite makes maintenance more annoying (e.g. they keep outdated usage patterns and buggy helpers alive)