search

JiaweiZhuang / xESMF

Universal Regridder for Geospatial Data
http://xesmf.readthedocs.io/
MIT License
269 stars 49 forks source link

ImportError: The ESMF shared library did not load properly. #54

Open tommylees112 opened 5 years ago

tommylees112 commented 5 years ago

The error

Traceback (most recent call last):
  File "/soge-home/users/chri4118/.conda/envs/crp/lib/python3.7/site-packages/ESMF/interface/loadESMF.py", line 118, in <module>
    mode=ct.RTLD_GLOBAL)
  File "/soge-home/users/chri4118/.conda/envs/crp/lib/python3.7/ctypes/__init__.py", line 356, in __init__
    self._handle = _dlopen(self._name, mode)
OSError: /soge-home/users/chri4118/.conda/envs/crp/lib/./libmpicxx.so.12: undefined symbol: MPIR_Keyval_set_proxy

----> 3 import xesmf as xe
      4 import numpy as np
      5

~/.conda/envs/crp/lib/python3.7/site-packages/xesmf/__init__.py in <module>
      1 from . import util
      2 from . import data
----> 3 from . frontend import Regridder

~/.conda/envs/crp/lib/python3.7/site-packages/xesmf/frontend.py in <module>
      7 import os
      8
----> 9 from . backend import (esmf_grid, add_corner,
     10                        esmf_regrid_build, esmf_regrid_finalize)
     11

~/.conda/envs/crp/lib/python3.7/site-packages/xesmf/backend.py in <module>
     17
     18 import numpy as np
---> 19 import ESMF
     20 import warnings
     21 import os

~/.conda/envs/crp/lib/python3.7/site-packages/ESMF/__init__.py in <module>
     68 #### IMPORT LIBRARIES #########################################################
     69
---> 70 from ESMF.api.esmpymanager import *
     71 from ESMF.api.grid import *
     72 from ESMF.api.mesh import *

~/.conda/envs/crp/lib/python3.7/site-packages/ESMF/api/esmpymanager.py in <module>
      9 from ESMF.api.constants import *
     10 from ESMF.util.exceptions import *
---> 11 from ESMF.interface.cbindings import *
     12 from ESMF.util.decorators import initialize
     13

~/.conda/envs/crp/lib/python3.7/site-packages/ESMF/interface/cbindings.py in <module>
     11 import ESMF.api.constants as constants
     12 from ESMF.util.decorators import deprecated, netcdf
---> 13 from ESMF.interface.loadESMF import _ESMF
     14
     15 def copy_struct(src):

~/.conda/envs/crp/lib/python3.7/site-packages/ESMF/interface/loadESMF.py in <module>
    119 except:
    120     traceback.print_exc(file=sys.stdout)
--> 121     raise ImportError('The ESMF shared library did not load properly.')

ImportError: The ESMF shared library did not load properly.

my environment

Conda list:

conda (crp) chri4118@linux1:/soge-home/projects/crop_yield/ml_drought$ conda list
# packages in environment at /soge-home/users/chri4118/.conda/envs/crp:
#
# Name                    Version                   Build  Channel
_libgcc_mutex             0.1                        main
affine                    2.2.2                      py_0    conda-forge
argparse                  1.4.0                     <pip>
asn1crypto                0.24.0                    <pip>
asn1crypto                0.24.0                   py37_0
atomicwrites              1.3.0                      py_0    conda-forge
atomicwrites              1.3.0                     <pip>
attrs                     19.1.0                     py_0    conda-forge
aws-sam-translator        1.11.0                    <pip>
aws-xray-sdk              2.4.2                     <pip>
backcall                  0.1.0                    py37_0
basemap                   1.2.0           py37h673bf1a_1001    conda-forge
basemap-data-hires        1.2.0                         0    conda-forge
bcrypt                    3.1.6                     <pip>
beautifulsoup4            4.7.1                     <pip>
blas                      1.0                         mkl
bleach                    3.1.0                    py37_0
bokeh                     1.0.4                    py37_0
boltons                   19.1.0                     py_0    conda-forge
boltons                   19.1.0                    <pip>
boost-cpp                 1.68.0            h11c811c_1000    conda-forge
boto                      2.49.0                    <pip>
boto3                     1.9.153                   <pip>
botocore                  1.12.153                  <pip>
Bottleneck                1.2.1                     <pip>
bottleneck                1.2.1           py37h3010b51_1001    conda-forge
bs4                       0.0.1                     <pip>
bzip2                     1.0.6             h14c3975_1002    conda-forge
ca-certificates           2018.11.29           ha4d7672_0    conda-forge
cachetools                3.0.0                     <pip>
cairo                     1.14.12              h8948797_3
cartopy                   0.17.0           py37hbb7e04d_1
cdsapi                    0.1.4                     <pip>
certifi                   2019.6.16                py37_0    conda-forge
cffi                      1.12.3           py37h8022711_0    conda-forge
cfgrib                    0.9.7                      py_0    conda-forge
cfn-lint                  0.21.3                    <pip>
cftime                    1.0.3.4                   <pip>
cftime                    1.0.3.4          py37hdd07704_0
chardet                   3.0.4                    py37_1
chardet                   3.0.4                     <pip>
click                     7.0                        py_0    conda-forge
click-plugins             1.0.4                      py_0    conda-forge
cligj                     0.5.0                      py_0    conda-forge
climate-indices           1.0.4                     <pip>
cloudpickle               0.6.1                    py37_0
colorcet                  1.0.0                    py37_0
cryptography              2.3.1            py37hc365091_0
cryptography              2.4.2                     <pip>
cudatoolkit               10.0.130                      0
curl                      7.64.1               hf8cf82a_0    conda-forge
cycler                    0.10.0                   py37_0
Cython                    0.29.3                    <pip>
cytoolz                   0.9.0.1          py37h14c3975_1
dask                      1.2.2                      py_2    conda-forge
dask-core                 1.2.2                      py_0    conda-forge
datashader                0.6.8                      py_0    pyviz
datashape                 0.5.4                    py37_1
dbus                      1.13.2               h714fa37_1
decorator                 4.3.0                    py37_0
descartes                 1.1.0                      py_2    conda-forge
dill                      0.2.9                     <pip>
distributed               1.28.1                   py37_0    conda-forge
docker                    4.0.1                     <pip>
docutils                  0.14                      <pip>
earthengine-api           0.1.164                   <pip>
eccodes                   2.12.3               h1ba12e4_2    conda-forge
ecdsa                     0.13.2                    <pip>
entrypoints               0.3                      py37_0
esmf                      7.1.0r            hdfb41a0_1003    conda-forge
esmpy                     7.1.0r           py37h24bf2e0_2    conda-forge
expat                     2.2.5                he0dffb1_0
ffmpeg                    3.1.3                         0    menpo
fiona                     1.8.4           py37h1c6dbfb_1002    conda-forge
fire                      0.1.3                      py_1    conda-forge
flake8                    3.7.7                    py37_0    conda-forge
fontconfig                2.13.1            h2176d3f_1000    conda-forge
freetype                  2.9.1                h8a8886c_1
freexl                    1.0.5                h14c3975_0
future                    0.17.1                    <pip>
gdal                      2.4.0           py37heb36068_1001    conda-forge
geojson                   2.4.1                      py_0    conda-forge
geopandas                 0.4.0                      py_1    conda-forge
geoplot                   0.2.4                      py_0    conda-forge
geos                      3.7.1                he6710b0_0
geotiff                   1.4.3             h1105359_1000    conda-forge
gettext                   0.19.8.1             hd7bead4_3
giflib                    5.1.4                h14c3975_1
glib                      2.56.2            had28632_1001    conda-forge
gmp                       6.1.2                h6c8ec71_1
google-api-python-client  1.7.7                     <pip>
google-auth               1.6.2                     <pip>
google-auth-httplib2      0.0.3                     <pip>
gst-plugins-base          1.12.5            h3865690_1000    conda-forge
gstreamer                 1.12.5            h0cc0488_1000    conda-forge
hdf4                      4.2.13            h9a582f1_1002    conda-forge
hdf5                      1.10.5          nompi_h3c11f04_1100    conda-forge
heapdict                  1.0.0                    py37_2
holoviews                 1.11.2                    <pip>
holoviews                 1.11.0             pyh39e3cac_0    pyviz
httplib2                  0.12.0                    <pip>
icu                       58.2                 h9c2bf20_1
idna                      2.8                       <pip>
idna                      2.8                      py37_0
imageio                   2.4.1                    py37_0
inspect2                  0.1                       <pip>
intel-openmp              2019.1                      144
ipdb                      0.11                       py_2    conda-forge
ipykernel                 5.1.0            py37h39e3cac_0
ipython                   7.2.0            py37h39e3cac_0
ipython_genutils          0.2.0                    py37_0
ipywidgets                7.4.2                    py37_0
jasper                    1.900.1           h07fcdf6_1006    conda-forge
jedi                      0.13.2                   py37_0
Jinja2                    2.10.1                    <pip>
jinja2                    2.10                     py37_0
Jinja2                    2.10                      <pip>
jmespath                  0.9.4                     <pip>
joblib                    0.13.1                    <pip>
jpeg                      9c                h14c3975_1001    conda-forge
json-c                    0.13.1               h1bed415_0
jsondiff                  1.1.2                     <pip>
jsonpatch                 1.23                      <pip>
jsonpickle                1.2                       <pip>
jsonpointer               2.0                       <pip>
jsonschema                2.6.0                    py37_0
jupyter                   1.0.0                    py37_7
jupyter_client            5.2.4                    py37_0
jupyter_console           6.0.0                    py37_0
jupyter_core              4.4.0                    py37_0
kealib                    1.4.10            he7154bc_1002    conda-forge
kiwisolver                1.0.1            py37hf484d3e_0
krb5                      1.16.3            h05b26f9_1001    conda-forge
libaec                    1.0.4                hf484d3e_0    conda-forge
libboost                  1.67.0               h46d08c1_4
libcurl                   7.64.1               hda55be3_0    conda-forge
libcxx                    7.0.0             h6bb024c_1002    conda-forge
libdap4                   3.19.1            hd48c02d_1000    conda-forge
libedit                   3.1.20170329      hf8c457e_1001    conda-forge
libffi                    3.2.1             he1b5a44_1006    conda-forge
libgcc-ng                 9.1.0                hdf63c60_0
libgdal                   2.4.0             h982c1cc_1001    conda-forge
libgfortran-ng            7.3.0                hdf63c60_0
libiconv                  1.15                 h63c8f33_5
libkml                    1.3.0                h590aaf7_4
libnetcdf                 4.6.2             h056eaf5_1002    conda-forge
libpng                    1.6.37               hed695b0_0    conda-forge
libpq                     10.6              h13b8bad_1000    conda-forge
libsodium                 1.0.16               h1bed415_0
libspatialindex           1.8.5             hf484d3e_1003    conda-forge
libspatialite             4.3.0a              hb08deb6_19
libssh2                   1.8.2                h22169c7_2    conda-forge
libstdcxx-ng              8.2.0                hdf63c60_1
libtiff                   4.0.10            h2733197_1001
libuuid                   2.32.1            h14c3975_1000    conda-forge
libxcb                    1.13                 h1bed415_1
libxml2                   2.9.8                h26e45fe_1
libxslt                   1.1.33               h7d1a2b0_0
linecache2                1.0.0                     <pip>
llvm-meta                 7.0.0                         0    conda-forge
llvmlite                  0.27.0           py37hd408876_0
lmoments3                 1.0.4                     <pip>
locket                    0.2.0                    py37_1
lxml                      4.3.0            py37hefd8a0e_0
MarkupSafe                1.1.0                     <pip>
markupsafe                1.1.0            py37h7b6447c_0
matplotlib                3.0.2           py37h8a2030e_1001    conda-forge
matplotlib-base           3.0.2           py37h167e16e_1001    conda-forge
mccabe                    0.6.1                      py_1    conda-forge
mistune                   0.8.4            py37h7b6447c_0
mkl                       2019.1                      144
mkl_fft                   1.0.11           py37h14c3975_0    conda-forge
mkl_random                1.0.2            py37h637b7d7_2    conda-forge
mock                      3.0.5                     <pip>
more-itertools            7.0.0                     <pip>
more-itertools            4.3.0                 py37_1000    conda-forge
moto                      1.3.8                     <pip>
mpi                       1.0                       mpich    conda-forge
mpich                     3.2.1             h1c2f66e_1008    conda-forge
mpl-animation             0.1                       <pip>
msgpack-python            0.6.1            py37hfd86e86_1
multipledispatch          0.6.0                    py37_0
multiprocess              0.70.7                    <pip>
munch                     2.3.2                      py_0    conda-forge
nbconvert                 5.3.1                    py37_0
nbformat                  4.4.0                    py37_0
nco                       1.0.0                     <pip>
ncurses                   6.1               hf484d3e_1002    conda-forge
netcdf-fortran            4.4.5             hea25ff8_1000    conda-forge
netcdf4                   1.4.2            py37h808af73_0    anaconda
networkx                  2.2                      py37_1
ninja                     1.9.0            py37hfd86e86_0
notebook                  5.7.4                    py37_0
numba                     0.42.0           py37h962f231_0
numpy                     1.16.2           py37h7e9f1db_0
numpy-base                1.16.2           py37hde5b4d6_0
olefile                   0.46                     py37_0
openjpeg                  2.3.0                h05c96fa_1
openssl                   1.1.1b               h14c3975_1    conda-forge
osmnx                     0.8.2                     <pip>
owslib                    0.17.0                   py37_0
packaging                 18.0                     py37_0
pandas                    0.24.2           py37hb3f55d8_0    conda-forge
pandoc                    2.2.3.2                       0
pandocfilters             1.4.2                    py37_1
param                     1.8.2                     <pip>
param                     1.8.2                      py_0    pyviz
paramiko                  2.4.2                     <pip>
parso                     0.3.1                    py37_0
partd                     0.3.9                    py37_0
pathos                    0.2.3                     <pip>
patsy                     0.5.1                      py_0    conda-forge
pcre                      8.41                 hc27e229_1
pexpect                   4.6.0                    py37_0
pickleshare               0.7.5                    py37_0
pillow                    5.4.1            py37h34e0f95_0
pint                      0.9                      py37_2    conda-forge
pip                       19.1.1                   py37_0    conda-forge
pixman                    0.34.0               hceecf20_3
pluggy                    0.11.0                     py_0    conda-forge
poppler                   0.67.0            h2fc8fa2_1002    conda-forge
poppler-data              0.4.9                         0
postgresql                10.6              h66cca7a_1000    conda-forge
pox                       0.2.5                     <pip>
ppft                      1.6.4.9                   <pip>
proj4                     5.2.0                he6710b0_1
prometheus_client         0.5.0                    py37_0
prompt_toolkit            2.0.7                    py37_0
psutil                    5.4.8            py37h7b6447c_0
psycopg2                  2.7.6.1         py37hb7f436b_1000    conda-forge
ptyprocess                0.6.0                    py37_0
py                        1.8.0                      py_0    conda-forge
pyasn1                    0.4.5                     <pip>
pyasn1-modules            0.2.3                     <pip>
pycodestyle               2.5.0                      py_0    conda-forge
pycparser                 2.19                     py37_1    conda-forge
pycparser                 2.19                      <pip>
pyct                      0.4.6                      py_0    pyviz
pyct-core                 0.4.6                      py_0    pyviz
pyepsg                    0.4.0                    py37_0
pyflakes                  2.1.1                      py_0    conda-forge
pygments                  2.3.1                    py37_0
pykdtree                  1.3.1            py37hdd07704_2
PyNaCl                    1.3.0                     <pip>
pyopenssl                 18.0.0                   py37_0
pyparsing                 2.3.1                    py37_0
pyproj                    1.9.6           py37hc0953d3_1000    conda-forge
pyqt                      5.6.0            py37h22d08a2_6
pysal                     1.14.4.post2          py37_1001    conda-forge
pysheds                   0.2.6            py37h14c3975_0    conda-forge
pyshp                     1.2.12                   py37_0    anaconda
pysocks                   1.6.8                    py37_0
pytest                    4.4.1                     <pip>
pytest                    4.5.0                    py37_0    conda-forge
pytest-flake8             1.0.4                    py37_0    conda-forge
python                    3.7.3                h33d41f4_1    conda-forge
python-dateutil           2.8.0                      py_0    conda-forge
python-jose               3.0.1                     <pip>
pytorch                   1.1.0           py3.7_cuda10.0.130_cudnn7.5.1_0    pytorch
pytz                      2019.1                     py_0    conda-forge
pyviz-comms               0.7.0                     <pip>
pyviz_comms               0.7.0                      py_0    pyviz
pywavelets                1.0.1            py37hdd07704_0
pyyaml                    3.13             py37h14c3975_0
pyzmq                     17.1.2           py37he6710b0_2
qt                        5.6.2                hf70d934_9    conda-forge
qtconsole                 4.4.3                    py37_0
rasterio                  1.0.13                    <pip>
rasterio                  1.0.22           py37h5b3f9e8_0    conda-forge
rasterstats               0.13.0                    <pip>
readline                  8.0                  hf8c457e_0    conda-forge
requests                  2.21.0                   py37_0
responses                 0.10.6                    <pip>
rsa                       4.0                       <pip>
rtree                     0.8.3                 py37_1000    conda-forge
s3transfer                0.2.0                     <pip>
salem                     0.2.4                     <pip>
scikit-image              0.15.0                    <pip>
scikit-image              0.14.1           py37he6710b0_0
scikit-learn              0.20.2           py37hd81dba3_0
scipy                     1.2.1            py37h7c811a0_0
scipy                     1.3.0                     <pip>
seaborn                   0.9.0                      py_0    conda-forge
seaborn                   0.9.0                     <pip>
send2trash                1.5.0                    py37_0
setuptools                41.0.1                   py37_0    conda-forge
shap                      0.28.5                    <pip>
shapely                   1.6.4            py37h86c5351_0
simplejson                3.16.0                    <pip>
sip                       4.18.1           py37hf484d3e_2
six                       1.12.0                py37_1000    conda-forge
snuggs                    1.4.2                     <pip>
snuggs                    1.4.3                      py_0    conda-forge
sortedcontainers          2.1.0                    py37_0
soupsieve                 1.9.1                     <pip>
sqlalchemy                1.2.16          py37h14c3975_1000    conda-forge
sqlite                    3.28.0               hcee41ef_1    conda-forge
statsmodels               0.9.0           py37h3010b51_1000    conda-forge
tblib                     1.3.2                    py37_0
terminado                 0.8.1                    py37_1
testpath                  0.4.2                    py37_0
tk                        8.6.9             hed695b0_1002    conda-forge
toolz                     0.9.0                    py37_0
torchvision               0.2.2                      py_3    pytorch
tornado                   5.1.1            py37h7b6447c_0
tqdm                      4.30.0                    <pip>
traceback2                1.4.0                     <pip>
traitlets                 4.3.2                    py37_0
tzcode                    2018g             h14c3975_1001    conda-forge
unittest2                 1.1.0                     <pip>
uritemplate               3.0.0                     <pip>
urllib3                   1.24.1                   py37_0
urllib3                   1.24.1                    <pip>
wcwidth                   0.1.7                    py37_0
webencodings              0.5.1                    py37_1
websocket-client          0.56.0                    <pip>
Werkzeug                  0.15.4                    <pip>
wheel                     0.33.4                   py37_0    conda-forge
widgetsnbextension        3.4.2                    py37_0
wrapt                     1.11.1                    <pip>
xarray                    0.12.2                     py_0    conda-forge
xclim                     0.9b0                      py_0    conda-forge
xerces-c                  3.2.2                h780794e_0
xesmf                     0.1.1                     <pip>
xmltodict                 0.12.0                    <pip>
xz                        5.2.4             h14c3975_1001    conda-forge
yaml                      0.1.7                had09818_2
zeromq                    4.3.1                he6710b0_3
zict                      0.1.3                    py37_0
zlib                      1.2.11            h14c3975_1004    conda-forge
akrherz commented 5 years ago

Your env listing shows a mixture of conda-forge and defaults channel packages. See this. The recommendation is to place conda-forge channel before defaults and use the strict setting.

tommylees112 commented 5 years ago

Thank you so much for your rapid reply! I will give htat a go

pjpetersik commented 4 years ago

I run into the same problem. Interestingly, the ImportError disappears when I run import ESMF before every other import.