search

opencv / opencv-python

Automated CI toolchain to produce precompiled opencv-python, opencv-python-headless, opencv-contrib-python and opencv-contrib-python-headless packages.
https://pypi.org/project/opencv-python/
MIT License
4.52k stars 843 forks source link

Compile or Crosscompile opencv-python for armv7 #793

Open Corallo opened 1 year ago

Corallo commented 1 year ago

Expected behaviour

I am trying to follow the instructions to build wheels for an armv7 device.

Actual behaviour

The build fails at the stage of building wheels for cmake

#47 876.8     Building wheel for cmake (pyproject.toml): started
#47 878.1     Building wheel for cmake (pyproject.toml): finished with status 'error'
#47 878.2     error: subprocess-exited-with-error
#47 878.2 
#47 878.2     × Building wheel for cmake (pyproject.toml) did not run successfully.
#47 878.2     │ exit code: 1
#47 878.2     ╰─> [33 lines of output]
#47 878.2         Traceback (most recent call last):
#47 878.2           File "/tmp/pip-build-env-1v0hi9i5/overlay/local/lib/python3.10/dist-packages/skbuild/setuptools_wrap.py", line 621, in setup
#47 878.2             cmkr = cmaker.CMaker(cmake_executable)
#47 878.2           File "/tmp/pip-build-env-1v0hi9i5/overlay/local/lib/python3.10/dist-packages/skbuild/cmaker.py", line 151, in __init__
#47 878.2             self.cmake_version = get_cmake_version(self.cmake_executable)
#47 878.2           File "/tmp/pip-build-env-1v0hi9i5/overlay/local/lib/python3.10/dist-packages/skbuild/cmaker.py", line 106, in get_cmake_version
#47 878.2             raise SKBuildError(
#47 878.2 
#47 878.2 
#47 878.2             =============================DEBUG ASSISTANCE=============================
#47 878.2             If you are seeing a compilation error please try the following steps to
#47 878.2             successfully install cmake:
#47 878.2             1) Upgrade to the latest pip and try again. This will fix errors for most
#47 878.2                users. See: https://pip.pypa.io/en/stable/installing/#upgrading-pip
#47 878.2             2) If on Linux, with glibc < 2.12, you can set PIP_ONLY_BINARY=cmake in
#47 878.2                order to retrieve the last manylinux1 compatible wheel.
#47 878.2             3) If on Linux, with glibc < 2.12, you can cap "cmake<3.23" in your
#47 878.2                requirements in order to retrieve the last manylinux1 compatible wheel.
#47 878.2             4) Open an issue with the debug information that follows at
#47 878.2                https://github.com/scikit-build/cmake-python-distributions/issues
#47 878.2 
#47 878.2             Python: 3.10.6
#47 878.2             platform: Linux-5.15.0-57-generic-armv7l-with-glibc2.35
#47 878.2             glibc: glibc 2.35
#47 878.2             machine: armv7l
#47 878.2             bits: 32
#47 878.2             pip: 22.0.2
#47 878.2             setuptools: 59.6.0
#47 878.2             scikit-build: 0.16.6
#47 878.2             PEP517_BUILD_BACKEND=setuptools.build_meta
#47 878.2             =============================DEBUG ASSISTANCE=============================
#47 878.2 
#47 878.2         Problem with the CMake installation, aborting build. CMake executable is cmake
#47 878.2         [end of output]
#47 878.2 
#47 878.2     note: This error originates from a subprocess, and is likely not a problem with pip.
#47 878.2     ERROR: Failed building wheel for cmake
#47 878.2   Successfully built numpy
#47 878.2   Failed to build cmake
#47 878.2   ERROR: Could not build wheels for cmake, which is required to install pyproject.toml-based projects
#47 878.7   error: subprocess-exited-with-error
#47 878.7   
#47 878.7   × pip subprocess to install build dependencies did not run successfully.
#47 878.7   │ exit code: 1
#47 878.7   ╰─> See above for output.
#47 878.7   
#47 878.7   note: This error originates from a subprocess, and is likely not a problem with pip.
#47 878.7   full command: /usr/bin/python3 /tmp/pip-standalone-pip-9zhxus_5/__env_pip__.zip/pip install --ignore-installed --no-user --prefix /tmp/pip-build-env-szb9nqwx/overlay --no-warn-script-location --no-binary :none: --only-binary :none: -i https://pypi.org/simple -- setuptools==59.2.0 wheel==0.37.0 'cmake>=3.1' pip 'scikit-build>=0.13.2' 'numpy==1.13.3; python_version=='"'"'3.6'"'"' and platform_machine != '"'"'aarch64'"'"' and platform_machine != '"'"'arm64'"'"'' 'numpy==1.17.0; python_version=='"'"'3.7'"'"' and platform_machine != '"'"'aarch64'"'"' and platform_machine != '"'"'arm64'"'"'' 'numpy==1.17.3; python_version=='"'"'3.8'"'"' and platform_machine != '"'"'aarch64'"'"' and platform_machine != '"'"'arm64'"'"'' 'numpy==1.19.3; python_version<='"'"'3.9'"'"' and sys_platform == '"'"'linux'"'"' and platform_machine == '"'"'aarch64'"'"'' 'numpy==1.21.0; python_version<='"'"'3.9'"'"' and sys_platform == '"'"'darwin'"'"' and platform_machine == '"'"'arm64'"'"'' 'numpy==1.19.3; python_version=='"'"'3.9'"'"' and platform_machine != '"'"'aarch64'"'"' and platform_machine != '"'"'arm64'"'"'' 'numpy==1.21.2; python_version>='"'"'3.10'"'"' and platform_system!='"'"'Darwin'"'"'' 'numpy==1.21.4; python_version>='"'"'3.10'"'"' and platform_system=='"'"'Darwin'"'"''
#47 878.7   cwd: [inherit]
#47 878.7   Installing build dependencies: finished with status 'error'
#47 878.7 error: subprocess-exited-with-error
#47 878.7 
#47 878.7 × pip subprocess to install build dependencies did not run successfully.
#47 878.7 │ exit code: 1
#47 878.7 ╰─> See above for output.
#47 878.7 
#47 878.7 note: This error originates from a subprocess, and is likely not a problem with pip.

Do you have any suggestions on how to build wheels for armv7?

Steps to reproduce

Issue submission checklist
asmorkalov commented 1 year ago

OpenCV is written in C++ and uses CMake for project configuration. Python bindings utilize Python API for native modules and are generated as C++ code too. To get OpenCV on your system you need to install CMake tool first with apt-get install cmake or with pip itself: pip install cmake.

Corallo commented 1 year ago

I've installed cmake with apt install cmake and confirmed that I have the version 3.22.1, but during the build of the wheels, it still tries to build the wheels of numpy and cmake, failing to install cmake wheels. The same happens when trying to install cmake with pip instead of apt. Is there some flag that I need to set to make opencv notice that cmake is already installed and doesn't need to build the wheels for that? Couldn't find anyone with similar problem online.

The failure comes from a failed test: 107/654 Test #80: BundleUtilities .............................................***Failed 1.13 sec that can't find a library:

 /tmp/pip-install-brzg9mx2/cmake_1e02609438ab4f2faad2c0c0318a1081/_skbuild/linux-aarch64-3.10/cmake-build/CMakeProject-build/Tests/BundleUtilities/testdir1/testbundleutils1:
error while loading shared libraries: libshared.so: cannot open shared
 object file: No such file or directory
asmorkalov commented 1 year ago

Hello @Corallo. OpenCV pyproject.toml includes cmake as dependency to ensure that all build per-requsites are there. Cmake team does not provide armv7 binary with pip (https://pypi.org/project/cmake/) and it's built automatically from sources. I have not found any other solution then just remove cmake from pyproject.toml. It's not universal solution as it breaks all other platforms.

Corallo commented 1 year ago

In this way installing cmake with apt-get will be enough?

I have tried to remove it from pyproject.toml, but doing so the build fails with a less clear error:

> [stage-1 37/37] RUN pip wheel .:                                                                                                                                                                        [20/673]
Processing /code/opencv-python 
Installing build dependencies: started
Installing build dependencies: still running... 
installing build dependencies: still running... 
Installing build dependencies: finished with status 'done'
Getting requirements to build wheel: started      
Getting requirements to build wheel: finished with status 'error'   
error: subprocess-exited-with-error 

× Getting requirements to build wheel did not run successfully.
│ exit code: 1
╰─> [25 lines of output]
[present-rich] git submodule update --init --recursive opencv exited with 1
Traceback (most recent call last):
 File "/root/.pyenv/versions/3.10.1/lib/python3.10/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 353, in <module>
#0 165.9           main() 
#0 165.9         File "/root/.pyenv/versions/3.10.1/lib/python3.10/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 335, in main
#0 165.9           json_out['return_val'] = hook(**hook_input['kwargs'])
#0 165.9         File "/root/.pyenv/versions/3.10.1/lib/python3.10/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 118, in get_requires_for_build_wheel
#0 165.9           return hook(config_settings)                                                          
#0 165.9         File "/tmp/pip-build-env-1i1h1um3/overlay/lib/python3.10/site-packages/setuptools/build_meta.py", line 162, in get_requires_for_build_wheel
#0 165.9           return self._get_build_requires(
#0 165.9         File "/tmp/pip-build-env-1i1h1um3/overlay/lib/python3.10/site-packages/setuptools/build_meta.py", line 143, in _get_build_requires
#0 165.9           self.run_setup()
#0 165.9         File "/tmp/pip-build-env-1i1h1um3/overlay/lib/python3.10/site-packages/setuptools/build_meta.py", line 267, in run_setup
#0 165.9           super(_BuildMetaLegacyBackend,
#0 165.9         File "/tmp/pip-build-env-1i1h1um3/overlay/lib/python3.10/site-packages/setuptools/build_meta.py", line 158, in run_setup
#0 165.9           exec(compile(code, __file__, 'exec'), locals())
#0 165.9         File "setup.py", line 494, in <module>
#0 165.9           main()
#0 165.9         File "setup.py", line 49, in main
#0 165.9           g.run_command(
#0 165.9         File "/tmp/pip-build-env-1i1h1um3/overlay/lib/python3.10/site-packages/pip/_internal/vcs/versioncontrol.py", line 650, in run_command
#0 165.9           return call_subprocess(
#0 165.9         File "/tmp/pip-build-env-1i1h1um3/overlay/lib/python3.10/site-packages/pip/_internal/utils/subprocess.py", line 224, in call_subprocess
#0 165.9           raise error
#0 165.9       pip._internal.exceptions.InstallationSubprocessError: git submodule update --init --recursive opencv exited with 1
#0 165.9       [end of output]
#0 165.9   
#0 165.9   note: This error originates from a subprocess, and is likely not a problem with pip.
#0 165.9 error: subprocess-exited-with-error

Is this how the pyproject.toml should look without cmake?

[build-system]
requires = [
  "setuptools==59.2.0", "wheel==0.37.0", "pip",
  "scikit-build>=0.13.2",
  "numpy==1.13.3; python_version=='3.6' and platform_machine != 'aarch64' and platform_machine != 'arm64'",
  "numpy==1.17.0; python_version=='3.7' and platform_machine != 'aarch64' and platform_machine != 'arm64'",
  "numpy==1.17.3; python_version=='3.8' and platform_machine != 'aarch64' and platform_machine != 'arm64'",
  "numpy==1.19.3; python_version<='3.9' and sys_platform == 'linux' and platform_machine == 'aarch64'",
  "numpy==1.21.0; python_version<='3.9' and sys_platform == 'darwin' and platform_machine == 'arm64'",
  "numpy==1.19.3; python_version=='3.9' and platform_machine != 'aarch64' and platform_machine != 'arm64'",
  "numpy==1.21.2; python_version=='3.10' and platform_system!='Darwin'",
  "numpy==1.21.4; python_version=='3.10' and platform_system=='Darwin'",
  "numpy==1.22.0; python_version>='3.11'"
]
asmorkalov commented 1 year ago

M.b. related: https://github.com/opencv/opencv-python/pull/820

sergregory commented 1 year ago

In case you need prebuilt opencv-python for raspberry pi: https://www.piwheels.org/project/opencv-python/

Corallo commented 1 year ago

@sergregory Thank you, but I need to build the wheels by myself, because I need to apply a small patch to opencv before building, to add an extra video-source api.