search

cern-fts / gfal2-python

python2 and 3 bindings for gfal2
https://dmc-docs.web.cern.ch/dmc-docs/gfal2-python.html
Apache License 2.0
5 stars 3 forks source link

Installation problem when building a docker image #5

Closed elehcim closed 9 months ago

elehcim commented 1 year ago

I'm trying to install gfal2-python in a docker image.

My Dockerfile is the following:

FROM python:3.11
RUN apt update && apt install --no-install-recommends -y cmake \
 gfal2 libgfal2-dev libgfal2-2 && rm -rf /var/lib/apt/lists/*
RUN pip install gfal2-python

I get the following error:

 > [5/9] RUN pip install gfal2-python:                                                                                                                                                                             
1.216 Collecting gfal2-python                                                                                                                                                                                      
1.258   Downloading gfal2-python-1.12.0.tar.gz (41 kB)                                                                                                                                                             
1.268      ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 41.5/41.5 kB 4.5 MB/s eta 0:00:00                                                                                                                              
1.275   Preparing metadata (setup.py): started                                                                                                                                                                     
1.543   Preparing metadata (setup.py): finished with status 'done'
1.544 Building wheels for collected packages: gfal2-python
1.545   Building wheel for gfal2-python (setup.py): started
2.472   Building wheel for gfal2-python (setup.py): finished with status 'error'
2.483   error: subprocess-exited-with-error
2.483   
2.483   × python setup.py bdist_wheel did not run successfully.
2.483   │ exit code: 1
2.483   ╰─> [110 lines of output]
2.483       running bdist_wheel
2.483       running build
2.483       running build_ext
2.483       -- The C compiler identification is GNU 12.2.0
2.483       -- The CXX compiler identification is GNU 12.2.0
2.483       -- Detecting C compiler ABI info
2.483       -- Detecting C compiler ABI info - done
2.483       -- Check for working C compiler: /usr/bin/cc - skipped
2.483       -- Detecting C compile features
2.483       -- Detecting C compile features - done
2.483       -- Detecting CXX compiler ABI info
2.483       -- Detecting CXX compiler ABI info - done
2.483       -- Check for working CXX compiler: /usr/bin/c++ - skipped
2.483       -- Detecting CXX compile features
2.483       -- Detecting CXX compile features - done
2.483       CMake Deprecation Warning at CMakeLists.txt:3 (cmake_minimum_required):
2.483         Compatibility with CMake < 2.8.12 will be removed from a future version of
2.483         CMake.
2.483       
2.483         Update the VERSION argument <min> value or use a ...<max> suffix to tell
2.483         CMake that the project does not need compatibility with older versions.
2.483       
2.483       
2.483       cmake source dir : /tmp/pip-install-pk1oo01c/gfal2-python_f3199125f1644c0f8d58ea031fece52d
2.483       -- gfal2-bindings is used as APPLICATION_NAME
2.483       CMake Warning (dev) at /usr/share/cmake-3.25/Modules/FindPackageHandleStandardArgs.cmake:438 (message):
2.483         The package name passed to `find_package_handle_standard_args` (Python3)
2.483         does not match the name of the calling package (PythonEasy).  This can lead
2.483         to problems in calling code that expects `find_package` result variables
2.483         (e.g., `_FOUND`) to follow a certain pattern.
2.483       Call Stack (most recent call first):
2.483         cmake/modules/FindPythonEasy.cmake:93 (FIND_PACKAGE_HANDLE_STANDARD_ARGS)
2.483         CMakeLists.txt:23 (find_package)
2.483       This warning is for project developers.  Use -Wno-dev to suppress it.
2.483       
2.483       -- Found Python3: /usr/local/bin/python
2.483       CMake Warning (dev) at /usr/share/cmake-3.25/Modules/FindPackageHandleStandardArgs.cmake:438 (message):
2.483         The package name passed to `find_package_handle_standard_args`
2.483         (PythonCurrentVersion) does not match the name of the calling package
2.483         (PythonEasy).  This can lead to problems in calling code that expects
2.483         `find_package` result variables (e.g., `_FOUND`) to follow a certain
2.483         pattern.
2.483       Call Stack (most recent call first):
2.483         cmake/modules/FindPythonEasy.cmake:113 (FIND_PACKAGE_HANDLE_STANDARD_ARGS)
2.483         CMakeLists.txt:23 (find_package)
2.483       This warning is for project developers.  Use -Wno-dev to suppress it.
2.483       
2.483       -- Found PythonCurrentVersion: 3.11
2.483       CMake Warning (dev) at /usr/share/cmake-3.25/Modules/FindPackageHandleStandardArgs.cmake:438 (message):
2.483         The package name passed to `find_package_handle_standard_args`
2.483         (PythonCurrentInclude) does not match the name of the calling package
2.483         (PythonEasy).  This can lead to problems in calling code that expects
2.483         `find_package` result variables (e.g., `_FOUND`) to follow a certain
2.483         pattern.
2.483       Call Stack (most recent call first):
2.483         cmake/modules/FindPythonEasy.cmake:114 (FIND_PACKAGE_HANDLE_STANDARD_ARGS)
2.483         CMakeLists.txt:23 (find_package)
2.483       This warning is for project developers.  Use -Wno-dev to suppress it.
2.483       
2.483       -- Could NOT find PythonCurrentInclude (missing: PYTHON_INCLUDE_PATH)
2.483       CMake Warning (dev) at /usr/share/cmake-3.25/Modules/FindPackageHandleStandardArgs.cmake:438 (message):
2.483         The package name passed to `find_package_handle_standard_args`
2.483         (PythonCurrentLibs) does not match the name of the calling package
2.483         (PythonEasy).  This can lead to problems in calling code that expects
2.483         `find_package` result variables (e.g., `_FOUND`) to follow a certain
2.483         pattern.
2.483       Call Stack (most recent call first):
2.483         cmake/modules/FindPythonEasy.cmake:115 (FIND_PACKAGE_HANDLE_STANDARD_ARGS)
2.483         CMakeLists.txt:23 (find_package)
2.483       This warning is for project developers.  Use -Wno-dev to suppress it.
2.483       
2.483       -- Could NOT find PythonCurrentLibs (missing: PYTHON_LIBRARIES)
2.483       CMake Warning (dev) at /usr/share/cmake-3.25/Modules/FindPackageHandleStandardArgs.cmake:438 (message):
2.483         The package name passed to `find_package_handle_standard_args`
2.483         (PythonCurrentModsDir) does not match the name of the calling package
2.483         (PythonEasy).  This can lead to problems in calling code that expects
2.483         `find_package` result variables (e.g., `_FOUND`) to follow a certain
2.483         pattern.
2.483       Call Stack (most recent call first):
2.483         cmake/modules/FindPythonEasy.cmake:116 (FIND_PACKAGE_HANDLE_STANDARD_ARGS)
2.483         CMakeLists.txt:23 (find_package)
2.483       This warning is for project developers.  Use -Wno-dev to suppress it.
2.483       
2.483       -- Could NOT find PythonCurrentModsDir (missing: PYTHON_SITE_PACKAGES)
2.483       CMake Warning at CMakeLists.txt:46 (message):
2.483         Boost Python3 library not found
2.483       
2.483       
2.483       -- Found PkgConfig: /usr/bin/pkg-config (found version "1.8.1")
2.483       -- Checking for module 'glib-2.0'
2.483       --   Found glib-2.0, version 2.74.6
2.483       -- GLIB2 libraries: glib-2.0
2.483       -- GLIB2 include dir: /usr/include/glib-2.0;/usr/lib/x86_64-linux-gnu/glib-2.0/include
2.483       -- Found GLIB2: glib-2.0
2.483       -- Checking for module 'gthread-2.0'
2.483       --   Found gthread-2.0, version 2.74.6
2.483       -- GTHREAD2 libraries: gthread-2.0;glib-2.0
2.483       -- GTHREAD2 include dir: /usr/include/glib-2.0;/usr/lib/x86_64-linux-gnu/glib-2.0/include
2.483       -- Found GTHREAD2: gthread-2.0;glib-2.0
2.483       -- Checking for module 'gfal2'
2.483       --   Found gfal2, version 2.21.3
2.483       -- Checking for module 'gfal_transfer'
2.483       --   Found gfal_transfer, version 2.21.3
2.483       -- GFAL2 libraries: gfal2;glib-2.0;gfal_transfer;gfal2;glib-2.0
2.483       -- GFAL2 include dir: /usr/include/gfal2;/usr/include/glib-2.0;/usr/lib/x86_64-linux-gnu/glib-2.0/include;/usr/include/gfal2;/usr/include/glib-2.0;/usr/lib/x86_64-linux-gnu/glib-2.0/include
2.483       -- Found GFAL2: gfal2;glib-2.0;gfal_transfer;gfal2;glib-2.0
2.483       -- Configuring done
2.483       -- Generating done
2.483       -- Build files have been written to: /tmp/pip-install-pk1oo01c/gfal2-python_f3199125f1644c0f8d58ea031fece52d/build/temp.linux-x86_64-cpython-311
2.483       error: [Errno 2] No such file or directory: 'build/temp.linux-x86_64-cpython-311/src/python3/gfal2.so'
2.483       [end of output]
2.483   
2.483   note: This error originates from a subprocess, and is likely not a problem with pip.
2.484   ERROR: Failed building wheel for gfal2-python
2.484   Running setup.py clean for gfal2-python
2.618 Failed to build gfal2-python
2.619 ERROR: Could not build wheels for gfal2-python, which is required to install pyproject.toml-based projects
2.703

Any idea why the gfal2.so library is not found? (I tried with python3.9 but I get the same error)

mpatrascoiu commented 1 year ago

Hello,

The gfal2.so library is the one you are trying to build via the gfal2-python project. After a successful build, it is usually installed in:

/usr/lib64/python3.6/site-packages/gfal2.so

From your CMake messages, I gather the following:

-- Could NOT find PythonCurrentInclude (missing: PYTHON_INCLUDE_PATH)
-- Could NOT find PythonCurrentLibs (missing: PYTHON_LIBRARIES)
-- Could NOT find PythonCurrentModsDir (missing: PYTHON_SITE_PACKAGES)
-- Boost Python3 library not found

Do you have python3.11-devel installed? Also, it looks like you're missing boost-python3 for Python3.11.

I suspect the underlying problem is running pip install gfal2-python without having the proper dependencies in place. I wrote a guide on what's needed to do a pip install here [1]

I'm currently fighting to change the PyPi description to link to this guide. This will make things easier, or at least, less confusing.

I noticed you're building on Debian-based distribution. Unfortunately, we don't offer much support for that as we provide for RHEL-based distributions. Any package dependency or building problems you might run into leave you in uncharted territory.

Hope this helps!

Cheers, Mihai

[1] https://dmc-docs.web.cern.ch/dmc-docs/gfal2-python/pip-install.html

mpatrascoiu commented 9 months ago

Hello,

Closing this issue. I believe running the pip-install guide would solve the problem.

Unfortunately, the build platform is Debian-based, which is not officially supported.

Cheers, Mihai

elehcim commented 7 months ago

Thank you, after a while I managed to work again on this issue and at the end following your directions I fixed the issue with the following Dockerfile:

FROM almalinux:9
RUN dnf install -y git epel-release
RUN git clone --depth 1 --branch v1.12.2 https://github.com/cern-fts/gfal2-python.git
WORKDIR /gfal2-python/
RUN ./ci/fedora-packages.sh
WORKDIR /gfal2-python/packaging/
RUN RPMBUILD_SRC_EXTRA_FLAGS="--without docs --without python2" make srpm
RUN dnf builddep -y gfal2-python-1.12.2-1.el9.src.rpm
RUN pip install gfal2-python
WORKDIR /

Cheers and thank you, Michele