eccodes 2.37.0 does no longer work with ecmwflibs (0.6.3)

ghaarsma commented 2 months ago

What happened?

When upgrading from eccodes 1.7.1 to eccodes 2.37.0, the package stopped working with ecmwflibs (Python 3.11 on Windows 10)

What are the steps to reproduce the bug?

(venv) PS C:\Users\gxvh\Python\cfgrib_test> python -m venv venv
(venv) PS C:\Users\gxvh\Python\cfgrib_test> .\venv\Scripts\Activate.ps1
(venv) PS C:\Users\gxvh\Python\cfgrib_test> pip install ecmwflibs eccodes==1.7.1

Collecting ecmwflibs
Collecting eccodes==1.7.1
Collecting numpy (from eccodes==1.7.1)
Collecting attrs (from eccodes==1.7.1)
Collecting cffi (from eccodes==1.7.1)
Collecting findlibs (from eccodes==1.7.1)
Collecting pycparser (from cffi->eccodes==1.7.1)
Building wheels for collected packages: findlibs
Successfully built findlibs
Installing collected packages: findlibs, pycparser, numpy, ecmwflibs, attrs, cffi, eccodes
Successfully installed attrs-24.2.0 cffi-1.17.1 eccodes-1.7.1 ecmwflibs-0.6.3 findlibs-0.0.5 numpy-2.1.1 pycparser-2.22

(venv) PS C:\Users\gxvh\Python\cfgrib_test> python
Python 3.11.9 (tags/v3.11.9:de54cf5, Apr  2 2024, 10:12:12) [MSC v.1938 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import eccodes
>>> exit()

(venv) PS C:\Users\gxvh\Python\cfgrib_test> pip install -U eccodes
Collecting eccodes
Installing collected packages: eccodes
  Attempting uninstall: eccodes
    Found existing installation: eccodes 1.7.1
    Uninstalling eccodes-1.7.1:
      Successfully uninstalled eccodes-1.7.1
Successfully installed eccodes-2.37.0

(venv) PS C:\Users\gxvh\Python\cfgrib_test> python
Python 3.11.9 (tags/v3.11.9:de54cf5, Apr  2 2024, 10:12:12) [MSC v.1938 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import eccodes
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "C:\Users\gxvh\Python\cfgrib_test\venv\Lib\site-packages\eccodes\__init__.py", line 13, in <module>
    from .eccodes import *  # noqa
    ^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\gxvh\Python\cfgrib_test\venv\Lib\site-packages\eccodes\eccodes.py", line 12, in <module>
    from gribapi import (
  File "C:\Users\gxvh\Python\cfgrib_test\venv\Lib\site-packages\gribapi\__init__.py", line 13, in <module>
    from .gribapi import *  # noqa
    ^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\gxvh\Python\cfgrib_test\venv\Lib\site-packages\gribapi\gribapi.py", line 34, in <module>
    from gribapi.errors import GribInternalError
  File "C:\Users\gxvh\Python\cfgrib_test\venv\Lib\site-packages\gribapi\errors.py", line 16, in <module>
    from .bindings import ENC, ffi, lib
  File "C:\Users\gxvh\Python\cfgrib_test\venv\Lib\site-packages\gribapi\bindings.py", line 106, in <module>
    raise RuntimeError("Cannot find the ecCodes library")
RuntimeError: Cannot find the ecCodes library
>>>

Version

2.37.0

Platform (OS and architecture)

Python 3.11.9, Windows 10, [MSC v.1938 64 bit (AMD64)]

Relevant log output

No response

Accompanying data

No response

Organisation

No response

iainrussell commented 1 month ago

Hi @ghaarsma,

Yes, we changed how we package the ecCodes binaries, and the new method does not (and will not) support Windows, only Linux and MacOS. Our Windows support will be phased out over time, but I can suggest the following, with the most desirable first:

try installing and using the WSL (Windows Subsystem for Linux)
try conda, which will give you the binaries and the Python bindings (the Windows build will also stop becoming available from here, but not yet)
stick with an older version of ecCodes
try setting environment variable LD_LIBRARY_PATH to the path where the ecCodes binary library is installed inside ecmwflibs

I hope one of these solutions will allow you to continue to work with ecCodes!

ghaarsma commented 1 month ago

Hi @iainrussell,

With Windows still being a very popular development platform, I'm hoping you (and everybody else at ECMWF) can reconsider.

It would only require 2 lines of code change in bindings.py to make it work again with ecmwflibs. I'll be happy to put together a PR for this change.

On the other hand, this would all be a moot point if ecmwflibs will not receive any future releases/upgrades. I know that @b8raoult maintains that project and you have contributed in the past. Wheels for Python 3.12 (and updated eccodes) would be greatly appreciated.

I'll be happy to help with testing (which I understand can be difficult if you don't have access to the required setup) and contribute where I can.

ecmwf / eccodes-python