Closed akuhnregnier closed 3 years ago
I have inadvertently reproduced this failure in the CI on my PR #3790 (see here for test output). Comparing this with #4112 where the equivalent tests pass, I had a look at the environment definitions using tkdiff
. The only difference I can see is the version of glibc
, which is v2.28 for the passing test, and v2.31 for the failing one. I note that @akuhnregnier's environment also has glibc 2.31.
Thanks @akuhnregnier for raising this!
Good spot @rcomer. I wonder if perhaps it's related to this change below, noted in the glibc v2.29 changelog.
- Optimized generic exp, exp2, log, log2, pow, sinf, cosf, sincosf and tanf.
Changing to use allclose
seems like a reasonable choice
Saying that, why are these not identical? This is the important bit of the test:
I don't know enough about the internals of iris, but surely this should be a passthrough of the __pow__
operation onto the cube data. Looking at what the test is doing -- i.e. comparing sqrt(cube) to sqrt(cube.data) -- even if the underlying numpy or glibc library implementation has changed, surely the change should be the same for both sides of this comparison.
@bjlittle might need your eyes on this one. My best guess is its something to do with a dtype casting here https://github.com/SciTools/iris/blob/055a60b06c63da917784ebff277d852f93a64968/lib/iris/analysis/maths.py#L498-L503
or that there is an implementation difference between np.power
and np.ndarray.__pow__
that results in a different code execution here https://github.com/SciTools/iris/blob/055a60b06c63da917784ebff277d852f93a64968/lib/iris/analysis/maths.py#L511-L512
We should probably just decide whether an AllClose test protects us from the errors that these tests are designed to catch, and if so change to that and move on with our day.
@pp-mo have you got any thoughts on the right way to proceed with this? should we change the code, or the test?
IMHO you should only test floating-point data for absolute equality for very special + rare reasons. But these tests are pretty ancient, and presumably written when we were less experienced!
Looking at the printed absolute and relative differences, they are really tiny.
So I wouldn't consider that significant.
So I for one would always use assertArrayAllClose or assertArrayNearlyEqual, and never assertArrayEqual.
surely this should be a passthrough of the pow operation onto the cube data.
Well, we obviously got away with it for a long time.
Possibly the reason there is any difference at all, is because the 'abs' operation used here is the Python built-in.
So that might deliver a different result from numpy.abs
? And maybe that behaviour is changing.
But I haven't manged to reproduce the error myself, so not entirely sure.
IMHO you should only test floating-point data for absolute equality for very special + rare reasons. But these tests are pretty ancient, and presumably written when we were less experienced!
Looking at the printed absolute and relative differences, they are really tiny. So I wouldn't consider that significant.
So I for one would always use assertArrayAllClose or assertArrayNearlyEqual, and never assertArrayEqual.
I think that makes a lot of sense!
surely this should be a passthrough of the pow operation onto the cube data.
Well, we obviously got away with it for a long time.
Possibly the reason there is any difference at all, is because the 'abs' operation used here is the Python built-in. So that might deliver a different result from
numpy.abs
? And maybe that behaviour is changing. But I haven't manged to reproduce the error myself, so not entirely sure.
I was wondering about that abs
too, but if I'm reading the code correctly, the check should be agnostic to the abs
, as e
is created after the abs call, so should have exactly the same bits.
Thanks @akuhnregnier for reporting this issue and apologies for taking so long to get it resolved. You saved us a ton of time debugging by providing so much detail, so thanks again! 🍻 on us if you're down in Exeter anytime soon.
Thanks! It's great to have been helpful, and thanks for the invite :smile:
🐛 Bug Report
The
test_square_root (iris.tests.test_basic_maths.TestExponentiate)
fails:How To Reproduce
Steps to reproduce the behaviour:
nox -s tests -- --verbose
Expected behaviour
No test failures.
Error log excerpts (including package versions)
Click to expand error log
``` nox > Running session tests-3.6 nox > Re-using existing conda env at .nox/tests-3-6. nox > python -m pip install --no-deps --editable . nox > conda info active environment : iris-dev active env location : /home/alexander/.pyenv/versions/miniconda3-latest/envs/iris-dev user config file : /home/alexander/.condarc populated config files : /home/alexander/.condarc conda version : 4.9.2 conda-build version : 3.18.11 python version : 3.7.3.final.0 virtual packages : __glibc=2.31=0 __unix=0=0 __archspec=1=x86_64 base environment : /home/alexander/.pyenv/versions/miniconda3-latest (writable) channel URLs : https://conda.anaconda.org/conda-forge/linux-64 https://conda.anaconda.org/conda-forge/noarch https://repo.anaconda.com/pkgs/main/linux-64 https://repo.anaconda.com/pkgs/main/noarch https://repo.anaconda.com/pkgs/r/linux-64 https://repo.anaconda.com/pkgs/r/noarch package cache : /home/alexander/.pyenv/versions/miniconda3-latest/pkgs /home/alexander/.conda/pkgs envs directories : /home/alexander/.pyenv/versions/miniconda3-latest/envs /home/alexander/.conda/envs platform : linux-64 user-agent : conda/4.9.2 requests/2.21.0 CPython/3.7.3 Linux/5.4.0-65-generic ubuntu/20.04.2 glibc/2.31 UID:GID : 1000:1000 netrc file : None offline mode : False nox > conda list --prefix=/home/alexander/Documents/PhD/reference_code/iris/.nox/tests-3-6 # packages in environment at /home/alexander/Documents/PhD/reference_code/iris/.nox/tests-3-6: # # Name Version Build Channel _libgcc_mutex 0.1 conda_forge conda-forge _openmp_mutex 4.5 1_gnu conda-forge alabaster 0.7.12 py_0 conda-forge antlr-python-runtime 4.7.2 py36h9f0ad1d_1002 conda-forge appdirs 1.4.4 pyh9f0ad1d_0 conda-forge asv 0.4.2 py36hc4f0c31_2 conda-forge atk-1.0 2.36.0 h3371d22_4 conda-forge babel 2.9.0 pyhd3deb0d_0 conda-forge black 20.8b1 py_1 conda-forge bokeh 2.1.1 py36h9f0ad1d_0 conda-forge brotlipy 0.7.0 py36h8f6f2f9_1001 conda-forge bzip2 1.0.8 h7f98852_4 conda-forge c-ares 1.17.1 h36c2ea0_0 conda-forge ca-certificates 2020.12.5 ha878542_0 conda-forge cairo 1.16.0 h7979940_1007 conda-forge cartopy 0.18.0 py36hb73674a_8 conda-forge certifi 2020.12.5 py36h5fab9bb_1 conda-forge cf-units 2.1.4 py36h68bb277_2 conda-forge cffi 1.14.4 py36hc120d54_1 conda-forge cfgv 3.2.0 py_0 conda-forge cftime 1.2.1 py36h68bb277_1 conda-forge chardet 4.0.0 py36h5fab9bb_1 conda-forge click 7.1.2 pyh9f0ad1d_0 conda-forge cloudpickle 1.6.0 py_0 conda-forge colorama 0.4.4 pyh9f0ad1d_0 conda-forge contextvars 2.4 py_0 conda-forge cryptography 3.3.1 py36h0a59100_1 conda-forge curl 7.71.1 he644dc0_8 conda-forge cycler 0.10.0 py_2 conda-forge cytoolz 0.11.0 py36h8f6f2f9_3 conda-forge dask 2021.2.0 pyhd8ed1ab_0 conda-forge dask-core 2021.2.0 pyhd8ed1ab_0 conda-forge dataclasses 0.7 pyhe4b4509_6 conda-forge dbus 1.13.6 hfdff14a_1 conda-forge distlib 0.3.1 pyh9f0ad1d_0 conda-forge distributed 2021.2.0 py36h5fab9bb_0 conda-forge docutils 0.16 py36h5fab9bb_3 conda-forge editdistance 0.5.3 py36hc4f0c31_3 conda-forge esmf 8.0.1 mpi_mpich_h3cbecb6_102 conda-forge esmpy 8.0.1 mpi_mpich_py36hd8aeb20_102 conda-forge expat 2.2.10 h9c3ff4c_0 conda-forge filelock 3.0.12 pyh9f0ad1d_0 conda-forge flake8 3.8.4 py_0 conda-forge font-ttf-dejavu-sans-mono 2.37 hab24e00_0 conda-forge font-ttf-inconsolata 2.001 hab24e00_0 conda-forge font-ttf-source-code-pro 2.030 hab24e00_0 conda-forge font-ttf-ubuntu 0.83 hab24e00_0 conda-forge fontconfig 2.13.1 hba837de_1004 conda-forge fonts-conda-ecosystem 1 0 conda-forge fonts-conda-forge 1 0 conda-forge freetype 2.10.4 h0708190_1 conda-forge fribidi 1.0.10 h36c2ea0_0 conda-forge fsspec 0.8.5 pyhd8ed1ab_0 conda-forge gdk-pixbuf 2.42.2 h0c95a7a_2 conda-forge geos 3.8.1 he1b5a44_0 conda-forge gettext 0.19.8.1 h0b5b191_1005 conda-forge giflib 5.2.1 h36c2ea0_2 conda-forge glib 2.66.6 h9c3ff4c_3 conda-forge glib-tools 2.66.6 h9c3ff4c_3 conda-forge graphite2 1.3.13 h58526e2_1001 conda-forge graphviz 2.42.3 h6939c30_2 conda-forge gst-plugins-base 1.14.5 h0935bb2_2 conda-forge gstreamer 1.18.3 h3560a44_0 conda-forge gtk2 2.24.33 hab0c2f8_0 conda-forge gts 0.7.6 h64030ff_2 conda-forge harfbuzz 2.7.4 h5cf4720_0 conda-forge hdf4 4.2.13 h10796ff_1004 conda-forge hdf5 1.10.6 mpi_mpich_h996c276_1014 conda-forge heapdict 1.0.1 py_0 conda-forge icu 68.1 h58526e2_0 conda-forge identify 1.5.13 pyh44b312d_0 conda-forge idna 2.10 pyh9f0ad1d_0 conda-forge imagehash 4.2.0 pyhd8ed1ab_0 conda-forge imagesize 1.2.0 py_0 conda-forge immutables 0.14 py36h8f6f2f9_1 conda-forge importlib-metadata 3.4.0 py36h5fab9bb_0 conda-forge importlib_metadata 3.4.0 hd8ed1ab_0 conda-forge importlib_resources 5.1.0 py36h5fab9bb_0 conda-forge iris-sample-data 2.3.0 pyh9f0ad1d_0 conda-forge jinja2 2.11.3 pyh44b312d_0 conda-forge jpeg 9d h36c2ea0_0 conda-forge kiwisolver 1.3.1 py36h605e78d_1 conda-forge krb5 1.17.2 h926e7f8_0 conda-forge lcms2 2.12 hddcbb42_0 conda-forge ld_impl_linux-64 2.35.1 hea4e1c9_2 conda-forge libblas 3.9.0 8_openblas conda-forge libcblas 3.9.0 8_openblas conda-forge libclang 11.0.1 default_ha53f305_1 conda-forge libcurl 7.71.1 hcdd3856_8 conda-forge libedit 3.1.20191231 he28a2e2_2 conda-forge libev 4.33 h516909a_1 conda-forge libevent 2.1.10 hcdb4288_3 conda-forge libffi 3.3 h58526e2_2 conda-forge libgcc-ng 9.3.0 h2828fa1_18 conda-forge libgfortran-ng 9.3.0 hff62375_18 conda-forge libgfortran5 9.3.0 hff62375_18 conda-forge libglib 2.66.6 h1f3bc88_3 conda-forge libgomp 9.3.0 h2828fa1_18 conda-forge libiconv 1.16 h516909a_0 conda-forge liblapack 3.9.0 8_openblas conda-forge libllvm11 11.0.1 hf817b99_0 conda-forge libmo_unpack 3.1.2 hf484d3e_1001 conda-forge libnetcdf 4.7.4 mpi_mpich_hdef422e_7 conda-forge libnghttp2 1.43.0 h812cca2_0 conda-forge libopenblas 0.3.12 pthreads_h4812303_1 conda-forge libpng 1.6.37 h21135ba_2 conda-forge libpq 12.3 h255efa7_3 conda-forge libssh2 1.9.0 hab1572f_5 conda-forge libstdcxx-ng 9.3.0 h6de172a_18 conda-forge libtiff 4.2.0 hdc55705_0 conda-forge libtool 2.4.6 h58526e2_1007 conda-forge libuuid 2.32.1 h7f98852_1000 conda-forge libwebp 1.2.0 h3452ae3_0 conda-forge libwebp-base 1.2.0 h7f98852_0 conda-forge libxcb 1.13 h7f98852_1003 conda-forge libxkbcommon 1.0.3 he3ba5ed_0 conda-forge libxml2 2.9.10 h72842e0_3 conda-forge locket 0.2.0 py_2 conda-forge lz4-c 1.9.3 h9c3ff4c_0 conda-forge markupsafe 1.1.1 py36h8f6f2f9_3 conda-forge matplotlib 3.3.4 py36h5fab9bb_0 conda-forge matplotlib-base 3.3.4 py36hd391965_0 conda-forge mccabe 0.6.1 py_1 conda-forge mo_pack 0.2.0 py36h92226af_1005 conda-forge mpi 1.0 mpich conda-forge mpi4py 3.0.3 py36h7b8b12a_5 conda-forge mpich 3.4.1 h846660c_2 conda-forge msgpack-python 1.0.2 py36h605e78d_1 conda-forge mypy_extensions 0.4.3 py36h5fab9bb_3 conda-forge mysql-common 8.0.22 ha770c72_3 conda-forge mysql-libs 8.0.22 h935591d_3 conda-forge nc-time-axis 1.2.0 py_1 conda-forge ncurses 6.2 h58526e2_4 conda-forge netcdf-fortran 4.5.3 mpi_mpich_h7ad8bfe_1 conda-forge netcdf4 1.5.5.1 nompi_py36hc29086f_101 conda-forge nodeenv 1.5.0 pyh9f0ad1d_0 conda-forge nose 1.3.7 py_1006 conda-forge nspr 4.29 h9c3ff4c_1 conda-forge nss 3.61 hb5efdd6_0 conda-forge numpy 1.19.5 py36h2aa4a07_1 conda-forge olefile 0.46 pyh9f0ad1d_1 conda-forge openssl 1.1.1i h7f98852_0 conda-forge packaging 20.9 pyh44b312d_0 conda-forge pandas 1.1.5 py36h284efc9_0 conda-forge pango 1.42.4 h69149e4_5 conda-forge partd 1.1.0 py_0 conda-forge pathspec 0.8.1 pyhd3deb0d_0 conda-forge pcre 8.44 he1b5a44_0 conda-forge pillow 6.2.2 py36h8328e55_0 conda-forge pip 21.0.1 pyhd8ed1ab_0 conda-forge pixman 0.40.0 h36c2ea0_0 conda-forge pockets 0.9.1 py_0 conda-forge pre-commit 2.10.1 py36h5fab9bb_0 conda-forge proj 7.2.0 he47e99f_1 conda-forge psutil 5.8.0 py36h8f6f2f9_1 conda-forge pthread-stubs 0.4 h36c2ea0_1001 conda-forge pycodestyle 2.6.0 pyh9f0ad1d_0 conda-forge pycparser 2.20 pyh9f0ad1d_2 conda-forge pyflakes 2.2.0 pyh9f0ad1d_0 conda-forge pygments 2.7.4 pyhd8ed1ab_0 conda-forge pyke 1.1.1 pyhd8ed1ab_1004 conda-forge pyopenssl 20.0.1 pyhd8ed1ab_0 conda-forge pyparsing 2.4.7 pyh9f0ad1d_0 conda-forge pyqt 5.12.3 py36h5fab9bb_7 conda-forge pyqt-impl 5.12.3 py36h7ec31b9_7 conda-forge pyqt5-sip 4.19.18 py36hc4f0c31_7 conda-forge pyqtchart 5.12 py36h7ec31b9_7 conda-forge pyqtwebengine 5.12.1 py36h7ec31b9_7 conda-forge pyshp 2.1.3 pyh44b312d_0 conda-forge pysocks 1.7.1 py36h5fab9bb_3 conda-forge python 3.6.12 hffdb5ce_0_cpython conda-forge python-dateutil 2.8.1 py_0 conda-forge python-stratify 0.1.1 py36h92226af_1003 conda-forge python-xxhash 2.0.0 py36h8f6f2f9_1 conda-forge python_abi 3.6 1_cp36m conda-forge pytz 2021.1 pyhd8ed1ab_0 conda-forge pyugrid 0.3.1 py_2 conda-forge pywavelets 1.1.1 py36h92226af_3 conda-forge pyyaml 5.4.1 py36h8f6f2f9_0 conda-forge qt 5.12.9 h9d6b050_2 conda-forge readline 8.0 he28a2e2_2 conda-forge regex 2020.11.13 py36h8f6f2f9_1 conda-forge requests 2.25.1 pyhd3deb0d_0 conda-forge scipy 1.5.3 py36h9e8f40b_0 conda-forge scitools-iris 3.1.dev0 dev_0Environment
See above
Additional Notes
Note that
test_self_referencing_load_issue_3367 (iris.tests.integration.test_netcdf.TestSelfReferencingVarLoad)
also fails for Python3.7
and3.8
, which is due to the NumPy version exceeding1.20.0
for those environments, coupled with outdated code in netcdf4-python. This has already been addressed (see Unidata/netcdf4-python#1065), and should be available in the upcoming version1.5.6
. Update (16/02/2021):test_self_referencing_load_issue_3367 (iris.tests.integration.test_netcdf.TestSelfReferencingVarLoad)
no longer fails after upating tonetcdf4 1.5.6
.