Open fnattino opened 1 year ago
Heads up @stephenworsley. Even if we're deprecating the API in question, I know we're using ESMPy a lot going forwards so this might be behaviour to watch for.
Heads up @stephenworsley. Even if we're deprecating the API in question, I know we're using ESMPy a lot going forwards so this might be behaviour to watch for.
Thanks for the heads up, I believe iris-esmf-regrid is already using the ignore_degenerate=True
argument (see here) so I believe this should be fine over there.
🐛 Bug Report
Hello again :) . While running Iris tests on OS X, I get a segmentation fault error from one of the tests of the ESMPy regridding function. Maybe this is OS-related, since I see that the same version of ESMF/ESMPy (8.2.0) is used in the CI and seems to give no issue on Linux.
I also see that the target Iris function is deprecated and it will be removed from future releases, so this is likely not so relevant - I thought I would anyway raise the issue in case somebody else bumps in the same problem.
How To Reproduce
On OS X, running the
test_global_collapse
fromlib/iris/tests/experimental/regrid/test_regrid_conservative_via_esmpy.py
:gives me a segmentation fault
``` ================================================================ test session starts ================================================================= platform darwin -- Python 3.10.8, pytest-7.2.0, pluggy-1.0.0 rootdir: /Users/fnattino/Programs/iris, configfile: pyproject.toml plugins: xdist-3.1.0 collected 14 items / 13 deselected / 1 selected lib/iris/tests/experimental/regrid/test_regrid_conservative_via_esmpy.py Fatal Python error: Segmentation fault Current thread 0x00007ff857d594c0 (most recent call first): File "/opt/miniconda3/envs/iris-dev/lib/python3.10/site-packages/ESMF/interface/cbindings.py", line 2089 in ESMP_FieldRegridStore File "/opt/miniconda3/envs/iris-dev/lib/python3.10/site-packages/ESMF/api/regrid.py", line 184 in __init__ File "/opt/miniconda3/envs/iris-dev/lib/python3.10/site-packages/ESMF/util/decorators.py", line 81 in new_func File "/Users/fnattino/Programs/iris/lib/iris/experimental/regrid_conservative.py", line 296 in regrid_conservative_via_esmpy File "/Users/fnattino/Programs/iris/lib/iris/tests/experimental/regrid/test_regrid_conservative_via_esmpy.py", line 385 in test_global_collapse File "/opt/miniconda3/envs/iris-dev/lib/python3.10/unittest/case.py", line 549 in _callTestMethod File "/opt/miniconda3/envs/iris-dev/lib/python3.10/unittest/case.py", line 591 in run File "/opt/miniconda3/envs/iris-dev/lib/python3.10/unittest/case.py", line 650 in __call__ File "/opt/miniconda3/envs/iris-dev/lib/python3.10/site-packages/_pytest/unittest.py", line 330 in runtest File "/opt/miniconda3/envs/iris-dev/lib/python3.10/site-packages/_pytest/runner.py", line 167 in pytest_runtest_call File "/opt/miniconda3/envs/iris-dev/lib/python3.10/site-packages/pluggy/_callers.py", line 39 in _multicall File "/opt/miniconda3/envs/iris-dev/lib/python3.10/site-packages/pluggy/_manager.py", line 80 in _hookexec File "/opt/miniconda3/envs/iris-dev/lib/python3.10/site-packages/pluggy/_hooks.py", line 265 in __call__ File "/opt/miniconda3/envs/iris-dev/lib/python3.10/site-packages/_pytest/runner.py", line 260 inExpected behaviour
I see that the regridding in the test is expected to fail and to raise a
ValueError
within ESMF - however I get segmentation fault. ESMF also leaves the following log file, which I assume it's part of the expected behaviour:ESMF log file content
``` 20221207 114128.397 ERROR PET0 ~~~~~~~~~~~~~~~~~~~~ Degenerate Element Detected ~~~~~~~~~~~~~~~~~~~~ 20221207 114128.397 ERROR PET0 degenerate elem. id=1 20221207 114128.397 ERROR PET0 20221207 114128.397 ERROR PET0 degenerate elem. coords (lon [-180 to 180], lat [-90 to 90]) (x,y,z) 20221207 114128.397 ERROR PET0 ----------------------------------------------------------------- 20221207 114128.398 ERROR PET0 0 (0.000000, -90.000000) (0.000000, 0.000000, -1.000000) 20221207 114128.398 ERROR PET0 1 (0.000000, -90.000000) (0.000000, 0.000000, -1.000000) 20221207 114128.398 ERROR PET0 2 (0.000000, 90.000000) (0.000000, 0.000000, 1.000000) 20221207 114128.398 ERROR PET0 3 (0.000000, 90.000000) (0.000000, 0.000000, 1.000000) 20221207 114128.398 ERROR PET0 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 20221207 114128.398 ERROR PET0 ESMCI_Mesh_Regrid_Glue.C:206 ESMCI_regrid_create() Invalid argument - - Dst contains a cell which has corners close enough that the cell collapses to a line or point ```I have noticed that adding
ignore_degenerate=True
as argument toESMF.Regrid()
bypasses the problem by skipping the problematic cells. The test still fails (theValueError
is not raised ifignore_degenerate
is set toTrue
) but I don't get segmentation fault.Environment
conda list
(environment built fromrequirements/ci/iris.yml
file):Full environment here
``` # packages in environment at /opt/miniconda3/envs/iris-dev: # # Name Version Build Channel alabaster 0.7.12 py_0 conda-forge antlr-python-runtime 4.7.2 py310h2ec42d9_1003 conda-forge atk-1.0 2.38.0 h1d18e73_1 conda-forge attrs 22.1.0 pyh71513ae_1 conda-forge babel 2.11.0 pyhd8ed1ab_0 conda-forge beautifulsoup4 4.11.1 pyha770c72_0 conda-forge brotli 1.0.9 hb7f2c08_8 conda-forge brotli-bin 1.0.9 hb7f2c08_8 conda-forge brotlipy 0.7.0 py310h90acd4f_1005 conda-forge bzip2 1.0.8 h0d85af4_4 conda-forge c-ares 1.18.1 h0d85af4_0 conda-forge ca-certificates 2022.9.24 h033912b_0 conda-forge cairo 1.16.0 h904041c_1014 conda-forge cartopy 0.21.0 py310h578c2b2_3 conda-forge certifi 2022.9.24 pyhd8ed1ab_0 conda-forge cf-units 3.1.1 py310h936d966_2 conda-forge cffi 1.15.1 py310ha78151a_2 conda-forge cfgv 3.3.1 pyhd8ed1ab_0 conda-forge cftime 1.6.2 py310h936d966_1 conda-forge charset-normalizer 2.1.1 pyhd8ed1ab_0 conda-forge click 8.1.3 unix_pyhd8ed1ab_2 conda-forge cloudpickle 2.2.0 pyhd8ed1ab_0 conda-forge colorama 0.4.6 pyhd8ed1ab_0 conda-forge contourpy 1.0.6 py310ha23aa8a_0 conda-forge cryptography 38.0.4 py310hdd0c95c_0 conda-forge curl 7.86.0 h581aaea_1 conda-forge cycler 0.11.0 pyhd8ed1ab_0 conda-forge dask-core 2022.12.0 pyhd8ed1ab_0 conda-forge distlib 0.3.6 pyhd8ed1ab_0 conda-forge docutils 0.17.1 py310h2ec42d9_3 conda-forge esmf 8.2.0 mpi_mpich_hcd927a8_102 conda-forge esmpy 8.2.0 mpi_mpich_py310hb1b2199_101 conda-forge exceptiongroup 1.0.4 pyhd8ed1ab_0 conda-forge execnet 1.9.0 pyhd8ed1ab_0 conda-forge expat 2.5.0 hf0c8a7f_0 conda-forge filelock 3.8.2 pyhd8ed1ab_0 conda-forge font-ttf-dejavu-sans-mono 2.37 hab24e00_0 conda-forge font-ttf-inconsolata 3.000 h77eed37_0 conda-forge font-ttf-source-code-pro 2.038 h77eed37_0 conda-forge font-ttf-ubuntu 0.83 hab24e00_0 conda-forge fontconfig 2.14.1 h5bb23bf_0 conda-forge fonts-conda-ecosystem 1 0 conda-forge fonts-conda-forge 1 0 conda-forge fonttools 4.38.0 py310h90acd4f_1 conda-forge freetype 2.12.1 h3f81eb7_1 conda-forge fribidi 1.0.10 hbcb3906_0 conda-forge fsspec 2022.11.0 pyhd8ed1ab_0 conda-forge gdk-pixbuf 2.42.8 h3648f77_1 conda-forge geos 3.11.1 hf0c8a7f_0 conda-forge gettext 0.21.1 h8a4c099_0 conda-forge giflib 5.2.1 hbcb3906_2 conda-forge graphite2 1.3.13 h2e338ed_1001 conda-forge graphviz 6.0.2 hc51f7b9_0 conda-forge gtk2 2.24.33 h7c1209e_2 conda-forge gts 0.7.6 hccb3bdf_2 conda-forge harfbuzz 5.3.0 h08f8713_0 conda-forge hdf4 4.2.15 h7aa5921_5 conda-forge hdf5 1.12.2 mpi_mpich_hc154f39_0 conda-forge icu 70.1 h96cf925_0 conda-forge identify 2.5.9 pyhd8ed1ab_0 conda-forge idna 3.4 pyhd8ed1ab_0 conda-forge imagehash 4.3.1 pyhd8ed1ab_0 conda-forge imagesize 1.4.1 pyhd8ed1ab_0 conda-forge importlib-metadata 5.1.0 pyha770c72_0 conda-forge iniconfig 1.1.1 pyh9f0ad1d_0 conda-forge iris-sample-data 2.4.0 pyhd8ed1ab_0 conda-forge jinja2 3.1.2 pyhd8ed1ab_1 conda-forge jpeg 9e hac89ed1_2 conda-forge kiwisolver 1.4.4 py310ha23aa8a_1 conda-forge krb5 1.19.3 hb98e516_0 conda-forge lcms2 2.14 h90f4b2a_0 conda-forge lerc 4.0.0 hb486fe8_0 conda-forge libblas 3.9.0 16_osx64_openblas conda-forge libbrotlicommon 1.0.9 hb7f2c08_8 conda-forge libbrotlidec 1.0.9 hb7f2c08_8 conda-forge libbrotlienc 1.0.9 hb7f2c08_8 conda-forge libcblas 3.9.0 16_osx64_openblas conda-forge libcurl 7.86.0 h581aaea_1 conda-forge libcxx 14.0.6 hccf4f1f_0 conda-forge libdeflate 1.14 hb7f2c08_0 conda-forge libedit 3.1.20191231 h0678c8f_2 conda-forge libev 4.33 haf1e3a3_1 conda-forge libffi 3.4.2 h0d85af4_5 conda-forge libgd 2.3.3 h1e214de_3 conda-forge libgfortran 5.0.0 9_5_0_h97931a8_26 conda-forge libgfortran5 11.3.0 h082f757_26 conda-forge libglib 2.74.1 h4c723e1_1 conda-forge libiconv 1.17 hac89ed1_0 conda-forge liblapack 3.9.0 16_osx64_openblas conda-forge libmo_unpack 3.1.2 h0a44026_1001 conda-forge libnetcdf 4.8.1 mpi_mpich_hc6a5375_6 conda-forge libnghttp2 1.47.0 h5aae05b_1 conda-forge libopenblas 0.3.21 openmp_h429af6e_3 conda-forge libpng 1.6.39 ha978bb4_0 conda-forge librsvg 2.54.4 h3d48ba6_0 conda-forge libsqlite 3.40.0 ha978bb4_0 conda-forge libssh2 1.10.0 h47af595_3 conda-forge libtiff 4.4.0 hdb44e8a_4 conda-forge libtool 2.4.6 he49afe7_1008 conda-forge libwebp 1.2.4 hfa4350a_0 conda-forge libwebp-base 1.2.4 h775f41a_0 conda-forge libxcb 1.13 h0d85af4_1004 conda-forge libxml2 2.10.3 hb9e07b5_0 conda-forge libzip 1.9.2 h6db710c_1 conda-forge libzlib 1.2.13 hfd90126_4 conda-forge llvm-openmp 15.0.6 h61d9ccf_0 conda-forge locket 1.0.0 pyhd8ed1ab_0 conda-forge markupsafe 2.1.1 py310h90acd4f_2 conda-forge matplotlib 3.6.2 py310h2ec42d9_0 conda-forge matplotlib-base 3.6.2 py310he725631_0 conda-forge mo_pack 0.2.0 py310h936d966_1008 conda-forge mpi 1.0 mpich conda-forge mpi4py 3.1.4 py310hb3ae6ab_0 conda-forge mpich 4.0.3 hd33e60e_100 conda-forge munkres 1.1.4 pyh9f0ad1d_0 conda-forge nc-time-axis 1.4.1 pyhd8ed1ab_0 conda-forge ncurses 6.3 h96cf925_1 conda-forge netcdf-fortran 4.6.0 mpi_mpich_hddcf434_1 conda-forge netcdf4 1.6.0 nompi_py310h6892ea4_102 conda-forge nodeenv 1.7.0 pyhd8ed1ab_0 conda-forge numpy 1.23.5 py310h1b7c290_0 conda-forge openjpeg 2.5.0 h5d0d7b0_1 conda-forge openssl 3.0.7 hfd90126_1 conda-forge packaging 21.3 pyhd8ed1ab_0 conda-forge pandas 1.5.2 py310hecf8f37_0 conda-forge pango 1.50.12 h7fca291_0 conda-forge partd 1.3.0 pyhd8ed1ab_0 conda-forge pcre2 10.40 h1c4e4bc_0 conda-forge pillow 9.2.0 py310hffcf78b_3 conda-forge pip 22.3.1 pyhd8ed1ab_0 conda-forge pixman 0.40.0 hbcb3906_0 conda-forge platformdirs 2.5.2 pyhd8ed1ab_1 conda-forge pluggy 1.0.0 pyhd8ed1ab_5 conda-forge pockets 0.9.1 py_0 conda-forge pre-commit 2.20.0 py310h2ec42d9_1 conda-forge proj 9.1.0 hcbd9701_0 conda-forge psutil 5.9.4 py310h90acd4f_0 conda-forge pthread-stubs 0.4 hc929b4f_1001 conda-forge pycparser 2.21 pyhd8ed1ab_0 conda-forge pydata-sphinx-theme 0.8.1 pyhd8ed1ab_0 conda-forge pygments 2.13.0 pyhd8ed1ab_0 conda-forge pyopenssl 22.1.0 pyhd8ed1ab_0 conda-forge pyparsing 3.0.9 pyhd8ed1ab_0 conda-forge pyproj 3.4.0 py310h8c678d5_2 conda-forge pyshp 2.3.1 pyhd8ed1ab_0 conda-forge pysocks 1.7.1 pyha2e5f31_6 conda-forge pytest 7.2.0 pyhd8ed1ab_2 conda-forge pytest-xdist 3.1.0 pyhd8ed1ab_0 conda-forge python 3.10.8 he7542f4_0_cpython conda-forge python-dateutil 2.8.2 pyhd8ed1ab_0 conda-forge python-stratify 0.2.post0 py310h936d966_3 conda-forge python-xxhash 3.0.0 py310h90acd4f_2 conda-forge python_abi 3.10 3_cp310 conda-forge pytz 2022.6 pyhd8ed1ab_0 conda-forge pywavelets 1.3.0 py310h936d966_2 conda-forge pyyaml 6.0 py310h90acd4f_5 conda-forge readline 8.1.2 h3899abd_0 conda-forge requests 2.28.1 pyhd8ed1ab_1 conda-forge scipy 1.9.3 py310h240c617_2 conda-forge scitools-iris 3.5.0.dev6+dirty dev_0