search

InsightSoftwareConsortium / ITKCLEsperanto

ITK filters accelerated with OpenCL via [clEsperanto](https://clesperanto.github.io/).
Apache License 2.0
4 stars 3 forks source link

Linux Python Compile Failure #30

Closed tbirdso closed 2 years ago

tbirdso commented 2 years ago

Failure to compile for Linux Python CI:

-- Found OpenCL: /usr/lib64/libOpenCL.so (found version "2.2") 
CMake Error at /usr/share/cmake-3.23/Modules/FindPackageHandleStandardArgs.cmake:230 (message):
-- Configuring incomplete, errors occurred!
  Could NOT find Python (missing: Python_INCLUDE_DIRS Python_LIBRARIES
See also "/work/_skbuild/linux-x86_64-3.8/cmake-build/CMakeFiles/CMakeOutput.log".
  Development Development.Module Development.Embed) (found version "2.7.5")
See also "/work/_skbuild/linux-x86_64-3.8/cmake-build/CMakeFiles/CMakeError.log".
Call Stack (most recent call first):
  /usr/share/cmake-3.23/Modules/FindPackageHandleStandardArgs.cmake:594 (_FPHSA_FAILURE_MESSAGE)
  /usr/share/cmake-3.23/Modules/FindPython.cmake:561 (find_package_handle_standard_args)
  _skbuild/linux-x86_64-3.8/cmake-build/_deps/clci-src/CMakeLists.txt:16 (find_package)

  File "/opt/python/cp38-cp38/lib/python3.8/site-packages/skbuild/setuptools_wrap.py", line 544, in setup

    env = cmkr.configure(cmake_args,

  File "/opt/python/cp38-cp38/lib/python3.8/site-packages/skbuild/cmaker.py", line 221, in configure
    raise SKBuildError(
An error occurred while configuring with CMake.
  Command:
    "cmake" "/work" "-G" "Ninja" "-DCMAKE_INSTALL_PREFIX:PATH=/work/_skbuild/linux-x86_64-3.8/cmake-install" "-DPYTHON_EXECUTABLE:FILEPATH=/opt/python/cp38-cp38/bin/python" "-DPYTHON_VERSION_STRING:STRING=3.8.13" "-DPYTHON_INCLUDE_DIR:PATH=/opt/_internal/cpython-3.8.13/include/python3.8" "-DPYTHON_LIBRARY:FILEPATH=libpython3.8.a" "-DSKBUILD:BOOL=TRUE" "-DCMAKE_MODULE_PATH:PATH=/opt/python/cp38-cp38/lib/python3.8/site-packages/skbuild/resources/cmake" "-DITK_DIR:PATH=/work/ITK-cp38-cp38-manylinux_2_28_x64" "-DITK_USE_SYSTEM_SWIG:BOOL=ON" "-DWRAP_ITK_INSTALL_COMPONENT_IDENTIFIER:STRING=PythonWheel" "-DSWIG_EXECUTABLE:FILEPATH=/work/ITK-cp38-cp38-manylinux_2_28_x64/Wrapping/Generators/SwigInterface/swig/bin/swig" "-DCMAKE_CXX_COMPILER_TARGET:STRING=x86_64-linux-gnu" "-DBUILD_TESTING:BOOL=OFF" "-DPython3_EXECUTABLE:FILEPATH=/opt/python/cp38-cp38/bin/python" "-DPython3_INCLUDE_DIR:PATH=/opt/python/cp38-cp38/bin/../include/python3.8" "-DCMAKE_BUILD_TYPE:STRING=Release"

Notes

2022-09-02T18:53:20.4583802Z -- Generating done 2022-09-02T18:53:20.4685154Z CMake Warning: 2022-09-02T18:53:20.4685758Z Manually-specified variables were not used by the project: 2022-09-02T18:53:20.4685963Z 2022-09-02T18:53:20.4686046Z PYTHON_EXECUTABLE 2022-09-02T18:53:20.4686268Z PYTHON_INCLUDE_DIR 2022-09-02T18:53:20.4686498Z PYTHON_VERSION_STRING 2022-09-02T18:53:20.4686662Z 


- Check ITKPythonBuilds download to ensure version matches `itk-wheel-tag`
tbirdso commented 2 years ago

@thewtex Do you have any thoughts on why finding Python development libraries could fail when building with Dockcross?

It looks like this is failing both at the ITKCLEsperanto level and in its dependency CLIc-prototype. Note that the latter uses FindPython but still fails when I change to FindPython3 in a development branch.

In both cases CMake is able to find the Python interpreter but fails to find other "Development" components (i.e. libraries, include directories). I have inspected the container and confirmed that development files accompany the interpreter as expected.

ITK and its external modules all seem to rely on development components and are not universally failing; see recent successful build in https://github.com/InsightSoftwareConsortium/ITKVkFFTBackend/runs/8234343023?check_suite_focus=true. However, in that log and others it is common to see the CMake config printout:

Could NOT find Python3 (missing: Python3_LIBRARIES Development Development.Embed) (found version "3.7.13")

In that case it seems like ITK uses a workaround to still set the Python3 executable; however, I have tried applying this in a development branch for ITKCLEsperanto + CLIc_prototype without success.

tbirdso commented 2 years ago

Identified source of failure:

  1. Python library files are not present in docker image for Linux Python wheel builds;
  2. CLIc_prototype dependency requires Python library and include files for building when only the Python interpreter is actually used.

Submitted a resolution in https://github.com/clEsperanto/CLIc_prototype/pull/100.