ecmwf / cfgrib

A Python interface to map GRIB files to the NetCDF Common Data Model following the CF Convention using ecCodes
Apache License 2.0
407 stars 77 forks source link

Cannot find the ecCodes library #329

Closed sydomine closed 1 year ago

sydomine commented 1 year ago

Hi,

I am not a python pro and the following issue seems to be above my skills.

I am using Windows and installed eccodes/grib with Mamba (conda-forge) in a dedicated environment named airpython. I code in python using Spyder. I use eccodes through cfgrib through xarray. It worked easily in previous environments (conda install), but today it doesn't (mamba install). :-(

When importing eccodes (or cfgrib) I receive the following error: File ~\mambaforge\envs\airpython\lib\site-packages\gribapi\bindings.py:35 raise RuntimeError("Cannot find the ecCodes library")

I cannot get it to work in this environment, but conda gives me other issues so I have to use mamba instead. Please help!

When running "python -m eccodes selfcheck" everything looks good: Found: ecCodes v2.28.0. Library: ~\mambaforge\envs\airpython\Library\bin\eccodes.dll Definitions: /MEMFS/definitions Samples: /MEMFS/samples Your system is ready. (I checked manually and could find eccodes.dll in place)

bindings.py:33-35 library_path = findlibs.find("eccodes") if library_path is None: raise RuntimeError("Cannot find the ecCodes library")

"import findlibs" works fine in my environment, but findlibs.find("eccodes") returns nothing.

This leads to findlibs: https://github.com/ecmwf/findlibs/blob/c21f47400d3cb513b92d6b1c684582643282ee92/findlibs/__init__.py

Searching os.environ, I have no ECCODES_DIR, ECCODES_HOME, LD_LIBRARY_PATH or DYLD_LIBRARY_PATH variables. However, I have 'CONDA_ENV_PATH': '/mambaforge/envs/airpython' and 'CONDA_ENV_PYTHON': '\mambaforge\envs\airpython\python.exe'

findlibs obviously won't work but how does "python -m eccodes selfcheck" tells me everything is right and finds the library? I added ECCODES_HOME to my environment variables, pointing to a directory with lib/libeccodes.dll in it, but of course it does not work. I have no idea how to progress from here.

From ~\mambaforge\envs\airpython\conda-meta\eccodes-2.28.0-h75381da_0.json Everything looks good: "extracted_package_dir": "~\mambaforge\pkgs\eccodes-2.28.0-h75381da_0", "features": "", "files": [ "Library/bin/bufr_compare.exe", "Library/bin/bufr_compare_dir", "Library/bin/bufr_copy.exe", "Library/bin/bufr_count.exe", "Library/bin/bufr_dump.exe", "Library/bin/bufr_filter", "Library/bin/bufr_get.exe", "Library/bin/bufr_index_build.exe", "Library/bin/bufr_ls.exe", "Library/bin/bufr_set.exe", "Library/bin/codes_bufr_filter.exe", "Library/bin/codes_count.exe", "Library/bin/codes_info.exe", "Library/bin/codes_parser.exe", "Library/bin/codes_split_file.exe", "Library/bin/eccodes.dll", "Library/bin/eccodes_memfs.dll", "Library/bin/grib2ppm.exe", "Library/bin/grib_compare.exe", "Library/bin/grib_copy.exe", "Library/bin/grib_count.exe", "Library/bin/grib_dump.exe", "Library/bin/grib_filter.exe", "Library/bin/grib_get.exe", "Library/bin/grib_get_data.exe", "Library/bin/grib_histogram.exe", "Library/bin/grib_index_build.exe", "Library/bin/grib_ls.exe", "Library/bin/grib_merge.exe", "Library/bin/grib_set.exe", "Library/bin/grib_to_netcdf.exe", "Library/bin/gts_compare.exe", "Library/bin/gts_copy.exe", "Library/bin/gts_count.exe", "Library/bin/gts_dump.exe", "Library/bin/gts_filter.exe", "Library/bin/gts_get.exe", "Library/bin/gts_ls.exe", "Library/bin/metar_compare.exe", "Library/bin/metar_copy.exe", "Library/bin/metar_dump.exe", "Library/bin/metar_filter.exe", "Library/bin/metar_get.exe", "Library/bin/metar_ls.exe", "Library/bin/tigge_check.exe", "Library/include/eccodes.h", "Library/include/eccodes_config.h", "Library/include/eccodes_ecbuild_config.h", "Library/include/eccodes_version.h", "Library/include/eccodes_windef.h", "Library/include/grib_api.h", "Library/lib/cmake/eccodes/eccodes-config-version.cmake", "Library/lib/cmake/eccodes/eccodes-config.cmake", "Library/lib/cmake/eccodes/eccodes-import.cmake", "Library/lib/cmake/eccodes/eccodes-targets-release.cmake", "Library/lib/cmake/eccodes/eccodes-targets.cmake", "Library/lib/eccodes.lib", "Library/lib/eccodes_memfs.lib", "Library/lib/pkgconfig/eccodes.pc", ".nonadmin" ],

sydomine commented 1 year ago

(from mamba list) I also have cfgrib 0.9.10.3, build pyhd8ed1ab_0, channel conda-forge.

iainrussell commented 1 year ago

Hello @sydomine,

We can't offer much support for Windows installations. I'll admit that this looks strange - the strangest thing is that the selfcheck command works! I can see from findlibs that it assumes the library will be called 'libeccodes.dll', when in fact the library does not have the 'lib' at the start. Confusingly, I can also see that if 'ecmwflibs' is installed, this will be used to locate the library. Perhaps this is causing some of the confusion. It would be worth you printing out all environment variables in case there's something of use there.

This issue really belongs in the ECMWF support portal though: https://confluence.ecmwf.int/category/sc

You can create a ticket there, and more pairs of eyes will look at it!

Best regards, Iain

sydomine commented 1 year ago

Solved with very appreciated help from the ECMWF team:

eccodes was actually correctly installed in my environment, and worked out of the box. It only did not work correctly in my spyder/ipython console.

The issue may be related to something like that : https://github.com/spyder-ide/spyder/issues/14480 Or that: https://github.com/spyder-ide/spyder/pull/14565

This is a spyder-related issue after all, not an eccodes/cfgrib related issue. So I am closing the issue here.

iainrussell commented 1 year ago

Fantastic - thanks for sharing that.