search

ecmwf / eccodes-python

Python interface to the ecCodes GRIB/BUFR decoder/encoder
Apache License 2.0
112 stars 32 forks source link

Building the fast CFFI API? #8

Open marfel opened 4 years ago

marfel commented 4 years ago

Hi,

sorry, are the instructions for building the faster API still valid? Because on the ECMWF website https://confluence.ecmwf.int/display/ECC/ecCodes+installation it says "The Python 3 bindings are now built with CFFI and are packaged separately"?

If the instructions are still true and could lead to a dramatic speedup, I would definitely like to have them. But I don't understand what you mean by "clone the repo in the same folder as your ecCodes source tree"? I tried cloning to

eccodes-2.13.1-Source/eccodes-python and eccodes-2.13.1-Source/src/eccodes-python

but every time I get 

gcc -pthread -B /home/martin/.conda/envs/py3/compiler_compat -Wl,--sysroot=/ -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -I/home/martin/.conda/envs/py3/include/python3.6m -c gribapi/_bindings.c -o ./gribapi/_bindings.o
gribapi/_bindings.c:492:10: fatal error: eccodes.h: No such file or directory
 #include <eccodes.h>
          ^~~~~~~~~~~

Could you give me a hint, please? Thanks a lot, Martin

alexamici commented 4 years ago

@marfel you are right the documentation is not clear, the clone must be in the same folder as eccodes-2.13.1-Source:

$ ls
eccodes-2.13.1-Source/
eccodes-python/

I'll reword the section to make it clearer.

ejd commented 1 month ago

I took the following steps to try building the fast bindings but had no luck:

  1. download eccodes-2.36.0.tar.gz

  2. tar -xvf eccodes-2.36.0.tar.gz

  3. git clone https://github.com/ecmwf/eccodes-python.git (the resulting working copy is at e3a59403562a9158cce10e64896a0256bcf3e7ed)

  4. python3.8 -m venv eccodes-python-venv

    At this point, the current directory looks like this:

    (eccodes-python-venv) ~/Documents/projects$ ls
eccodes-2.36.0/
eccodes-python/
eccodes-python-venv/

  5. . eccodes-python-venv/bin/activate

  6. pip install "numpy<1.25" attrs cffi findlibs (per install_requires in setup.py)

  7. cd eccodes-python

  8. python builder.py

I get the same error as in https://github.com/ecmwf/eccodes-python/issues/8#issue-502537670 above. Here's the output from the invocation of python builder.py

generating ./gribapi/_bindings.c
(already up-to-date)
the current directory is '/home/ejd/Documents/projects/eccodes-python'
running build_ext
building 'gribapi._bindings' extension
x86_64-linux-gnu-gcc -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O2 -Wall -g -fstack-protector-strong -Wformat -Werror=format-security -g -fwrapv -O2 -g -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -fPIC -I/home/ejd/Documents/projects/eccodes-python-venv/include -I/usr/include/python3.8 -c gribapi/_bindings.c -o ./gribapi/_bindings.o
gribapi/_bindings.c:569:10: fatal error: eccodes.h: No such file or directory
  569 | #include <eccodes.h>
      |          ^~~~~~~~~~~
compilation terminated.