Cannot find the ecCodes library when pip installing eccodes 1.3.X

[pmav99]

Reproducer

conda create -y  --name test_eccodes python=3.8 pip eccodes
conda activate test_eccodes
python -mvenv venv
source venv/bin/activate
pip install eccodes==1.3.0 # or 1.3.1 or 1.3.2
python -m eccodes selfcheck

results in

Traceback (most recent call last):
  File "/home/panos/.conda/envs/test_eccodes/lib/python3.8/runpy.py", line 185, in _run_module_as_main
    mod_name, mod_spec, code = _get_module_details(mod_name, _Error)
  File "/home/panos/.conda/envs/test_eccodes/lib/python3.8/runpy.py", line 144, in _get_module_details
    return _get_module_details(pkg_main_name, error)
  File "/home/panos/.conda/envs/test_eccodes/lib/python3.8/runpy.py", line 111, in _get_module_details
    __import__(pkg_name)
  File "/home/panos/venv_eccodes/lib/python3.8/site-packages/eccodes/__init__.py", line 15, in <module>
    from .eccodes import *
  File "/home/panos/venv_eccodes/lib/python3.8/site-packages/eccodes/eccodes.py", line 12, in <module>
    from gribapi import __version__
  File "/home/panos/venv_eccodes/lib/python3.8/site-packages/gribapi/__init__.py", line 13, in <module>
    from .gribapi import *  # noqa
  File "/home/panos/venv_eccodes/lib/python3.8/site-packages/gribapi/gribapi.py", line 32, in <module>
    from .bindings import ENC, ffi, lib
  File "/home/panos/venv_eccodes/lib/python3.8/site-packages/gribapi/bindings.py", line 33, in <module>
    raise RuntimeError("Cannot find the ecCodes library")
RuntimeError: Cannot find the ecCodes library

Downgrading to 1.2.0 works just fine

$ pip install eccodes=1.2.0
$ python -m eccodes selfcheck

Found: ecCodes v2.21.0.
Your system is ready.

The eccodes package from conda is eccodes conda-forge/linux-64::eccodes-2.21.0-he2bb022_1

Cleaning up:

deactivate
conda deactivate
rm -rf venv
conda env remove --name test_eccodes

[Xunius]

Same here. pip install eccodes==1.2.0 worked for me.

[cccassidy]

Can confirm what the above posters found as well, works with 1.2.0 but fails with anything 1.3.x Using Python3.8 w/ ecCodes 2.22.0

[cccassidy]

Hi @pmav99 @Xunius I just did a fresh install of ecCodes 2.22.1 and eccodes-python 1.3.3 on Ubuntu 20.04 and the import worked fine. I saw v1.3.3 was released since we noticed this error maybe it was fixed in that version?

[cccassidy]

Tried some more tests and there appears to be some sort of conflict on RHEL-based systems: Amazon Linux 2 for example puts the libraries in a lib64/ directory (which fails) while Ubuntu does lib/ (which works).

As a workaround I renamed lib64 to lib and added the lib path to LD_LIBRARY_PATH and it worked under AL2. Not idea but maybe that helps track down the root cause?

[Balinus]

Tried some more tests and there appears to be some sort of conflict on RHEL-based systems: Amazon Linux 2 for example puts the libraries in a lib64/ directory (which fails) while Ubuntu does lib/ (which works).

As a workaround I renamed lib64 to lib and added the lib path to LD_LIBRARY_PATH and it worked under AL2. Not idea but maybe that helps track down the root cause?

Thanks! Updating the LD_LIBRARY_PATH is the only thing that worked for me (cluster on Cent OS)

[nfick]

Changing versions didn't work for me. All I had to do was: pip install ecmwflibs

From bindings.py in:

File "/home/panos/venv_eccodes/lib/python3.8/site-packages/gribapi/bindings.py", line 33, in <module>
    raise RuntimeError("Cannot find the ecCodes library")

There was:

try:
    import ecmwflibs as findlibs
except ImportError:
    import findlibs

The findlibs package was the problem.

[petebachant]

@nfick's fix works for me on Linux, but on Windows it causes some strange DLL load warnings:

C:\Users\peteb\miniconda3\envs\wedev\lib\site-packages\ecmwflibs\__init__.py:75:
 UserWarning: DLL load failed while importing _ecmwflibs: The specified procedur
e could not be found.
  warnings.warn(str(e))
C:\Users\peteb\miniconda3\envs\wedev\lib\site-packages\xarray\backends\cfgrib_.p
y:27: UserWarning: Failed to load cfgrib - most likely there is a problem access
ing the ecCodes library. Try `import cfgrib` to get the full error message
  warnings.warn(

But import eccodes works without error.

[CrayonAki]

Ubuntu 20.04, python 3.8 Following @nfick, I installed ecmwflibs with pip install ecmwflibs. I also tried to change version, but eccodes==1.2.0 seemed not working. So I tried version eccodes==1.3.1. I don't know which step is actually working, but finally eccodes can be imported.

[vipinkarthikeyan]

Before installing cfgrib, $ pip install ecmwflibs $ pip install eccodes==1.3.1 and then $ pip install cfgrib

my ecmwflibs version is 0.4.3

Thanks to @CrayonAki

[peterdudfield]

Im getting a similar error when runner a docker file

FROM python:3.9-slim

RUN apt-get -qq update \
    && apt-get -qq install -y --no-install-recommends \
        libproj-dev \
        libgeos-dev \
    && apt-get autoclean && apt-get autoremove \
    > /dev/null

# copy files
COPY metofficeamd/ app/metofficeamd/
COPY setup.py app/setup.py
COPY README.md app/README.md
COPY requirements.txt app/requirements.txt

# change to app folder
WORKDIR /app

# install library
RUN pip uninstall cfgrib
RUN pip install ecmwflibs
RUN pip install eccodes==1.3.1
RUN pip install cfgrib
RUN pip install -e .

# Example commnad that can be used, need to set API_KEY, API_SECRET and SAVE_DIR
CMD ["python", "metofficeamd/app.py"]

[peterdudfield]

Im getting a similar error when runner a docker file

FROM python:3.9-slim

RUN apt-get -qq update \
    && apt-get -qq install -y --no-install-recommends \
        libproj-dev \
        libgeos-dev \
    && apt-get autoclean && apt-get autoremove \
    > /dev/null

# copy files
COPY metofficeamd/ app/metofficeamd/
COPY setup.py app/setup.py
COPY README.md app/README.md
COPY requirements.txt app/requirements.txt

# change to app folder
WORKDIR /app

# install library
RUN pip uninstall cfgrib
RUN pip install ecmwflibs
RUN pip install eccodes==1.3.1
RUN pip install cfgrib
RUN pip install -e .

# Example commnad that can be used, need to set API_KEY, API_SECRET and SAVE_DIR
CMD ["python", "metofficeamd/app.py"]

by adding 'libeccodes-tools' it now seems to work

[epasseto]

What solved for me was REALLY to install ECMWFlibs, just because eccodes depends on this earlier library to work. The version of ECcodes does not matter!

So, at Google Colab, you can do: !$ pip install ecmwflibs !$ pip install eccodes !$ pip install cfgrib

Found: eccodes v2.26.0 Your system is ready

[djhoese]

I'm working with someone who is trying to add cfgrib/eccodes to our build environment and is running into weird behavior. Sometimes they get the same error(s) mentioned here and other times the errors are slightly different and sometimes it works. Whether the environment works or errors out seems to be related to the order the packages are installed.

The order the packages are installed shouldn't matter, right? There should be no reason you can't do pip install ecmwflibs eccodes cfgrib since all the usage happens at runtime not install time...right?

Note: Some of the issues we're running into may be specific to us trying to use pyinstaller to bundle our environment into a distributed application.

[observingClouds]

Just came across this. I think it would be very valuable if the eccodes library and the eccodes-python package would also be distinguishable when installing in a conda environment:

conda install eccodes  # installs eccodes library without python module
pip install eccodes # installs eccodes-python package

Because they have both the same name, conda list lists only the eccodes that has been installed last.

[josuemtzmo]

Perhaps adding to @observingClouds, I had the same issue and the way to fix it was to install in order ecmwflibs, eccodes and cfgrib. If I installed them in any different order, it created this issue.