search

kivy / kivy-ios

Toolchain for compiling Python / Kivy / other libraries for iOS
https://kivy.org/docs/guide/packaging-ios.html
MIT License
758 stars 237 forks source link

Build numpy: '['/kivy/dist/hostpython3/bin/python', '-m', 'cython', '-3', '--fast-fail', '-o', '_philox.c', '_philox.pyx']' returned non-zero exit status 1.' #835

Closed krs1w closed 10 months ago

krs1w commented 11 months ago

Versions

Describe the bug My kivy-setup for building python and numpy was working a few days ago, but fails as of today. Numpy shows some deprecation warnings and ultimately fails with '['/Users/fprefect/Developer/kivy/dist/hostpython3/bin/python', '-m', 'cython', '-3', '--fast-fail', '-o', '_philox.c', '_philox.pyx']' returned non-zero exit status 1. I have reproduced this error with a pip installed kivy-ios and by manual checkout, on M1 Mac mini and Intel MacBook. All prior recipes in the build run work flawlessly (hostopenssl, hostpython3, libffi, openssl, python3) producing libcrypto.a, libffi.a, libpython3.a, libssl.a. The log output looks kind of cut off at the end but this is the complete output of the console. I did cut some of the numpy log naming all files extracted but if you need any more logs or infos I'm happy to provide :)

To Reproduce

Expected behavior Working build, producing dist/lib/libnumpy.a

Logs

[notice] A new release of pip available: 22.3.1 -> 23.2 [notice] To update, run: pip install --upgrade pip

- toolchain build numpy

[INFO ] Download numpy [INFO ] Downloading https://pypi.python.org/packages/source/n/numpy/numpy-1.24.2.tar.gz [DEBUG ] New State: numpy.download at 2023-07-18 23:44:43.159955 [INFO ] Extract numpy [INFO ] Extract numpy for x86_64 [INFO ] Extract /Users/fprefect/Developer/kivy/.cache/numpy-numpy-1.24.2.tar.gz into /Users/fprefect/Developer/kivy/build/numpy/x86_64 [INFO ] Running Shell: /usr/bin/tar ('-C', '/Users/fprefect/Developer/kivy/build/numpy/x86_64', '-xv', '-z', '-f', '/Users/fprefect/Developer/kivy/.cache/numpy-numpy-1.24.2.tar.gz') {'_iter': True, '_out_bufsize': 1, '_err_to_out': True} [DEBUG ] x numpy-1.24.2/ [DEBUG ] x numpy-1.24.2/.coveragerc

[. . .]

[DEBUG ] x numpy-1.24.2/tox.ini [DEBUG ] x numpy-1.24.2/versioneer.py [DEBUG ] New State: numpy.extract at 2023-07-18 23:44:43.838770 [INFO ] Install_hostpython_prerequisites numpy [INFO ] Executing pip for hostpython with: ['install', 'Cython'] [INFO ] Running Shell: /Users/fprefect/Developer/kivy/dist/hostpython3/bin/pip3 ('install', 'Cython') {'_iter': True, '_out_bufsize': 1, '_err_to_out': True} [DEBUG ] Collecting Cython [DEBUG ] Using cached Cython-3.0.0-py2.py3-none-any.whl (1.2 MB) [DEBUG ] Installing collected packages: Cython [DEBUG ] Successfully installed Cython-3.0.0 [DEBUG ] [DEBUG ] [notice] A new release of pip available: 22.3.1 -> 23.2 [DEBUG ] [notice] To update, run: /Users/fprefect/Developer/kivy/dist/hostpython3/bin/python3.10 -m pip install --upgrade pip [DEBUG ] New State: numpy.install_hostpython_prerequisites at 2023-07-18 23:44:46.033334 [INFO ] Build_all numpy [INFO ] Build numpy for x86_64, arm64 (filtered) [INFO ] Build numpy [INFO ] Prebuild numpy for x86_64 [INFO ] Apply patch skip-math-test.patch [INFO ] Apply patch duplicated_symbols.patch [INFO ] Build numpy for x86_64 [INFO ] Running Shell: /Users/fprefect/Developer/kivy/dist/hostpython3/bin/python ('setup.py', 'build_ext', '-g') {'_env': {'CC': '/var/folders/k8/h2f5jsvx7pd6dzrjjn58ph0r0000gn/T/tmp3xscqtl5 -O3 -miphoneos-version-min=9.0 -I/Users/fprefect/Developer/kivy/dist/include/x86_64/ffi -I/Users/fprefect/Developer/kivy/dist/include/x86_64/openssl -I/Users/fprefect/Developer/kivy/dist/include/common/numpy -I/Users/fprefect/Developer/kivy/dist/include/x86_64', 'CXX': '/var/folders/k8/h2f5jsvx7pd6dzrjjn58ph0r0000gn/T/tmp629f_x6f', 'AR': '/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/ar', 'LD': '/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/ld', 'OTHER_CFLAGS': '-I/Users/fprefect/Developer/kivy/dist/include/x86_64/ffi -I/Users/fprefect/Developer/kivy/dist/include/x86_64/openssl -I/Users/fprefect/Developer/kivy/dist/include/common/numpy -I/Users/fprefect/Developer/kivy/dist/include/x86_64', 'OTHER_LDFLAGS': '-L/Users/fprefect/Developer/kivy/dist/lib', 'CFLAGS': '-O3 -miphoneos-version-min=9.0 -I/Users/fprefect/Developer/kivy/dist/include/x86_64/ffi -I/Users/fprefect/Developer/kivy/dist/include/x86_64/openssl -I/Users/fprefect/Developer/kivy/dist/include/common/numpy -I/Users/fprefect/Developer/kivy/dist/include/x86_64', 'LDFLAGS': '-arch x86_64 -L/Users/fprefect/Developer/kivy/dist/lib -L/Applications/Xcode.app/Contents/Developer/Platforms/iPhoneSimulator.platform/Developer/SDKs/iPhoneSimulator16.4.sdk/usr/lib -miphoneos-version-min=9.0', 'KIVYIOSROOT': '/Users/fprefect/Developer/kivy/venv/lib/python3.10/site-packages/kivy_ios', 'IOSSDKROOT': '/Applications/Xcode.app/Contents/Developer/Platforms/iPhoneSimulator.platform/Developer/SDKs/iPhoneSimulator16.4.sdk', 'CUSTOMIZED_OSX_COMPILER': 'True', 'LDSHARED': '/Users/fprefect/Developer/kivy/venv/lib/python3.10/site-packages/kivy_ios/tools/liblink', 'ARM_LD': '/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/ld', 'ARCH': 'x86_64', 'NPY_BLAS_ORDER': '', 'NPY_LAPACK_ORDER': ''}, '_iter': True, '_out_bufsize': 1, '_err_to_out': True} [DEBUG ] Running from numpy source directory. [DEBUG ] /Users/fprefect/Developer/kivy/build/numpy/x86_64/numpy-1.24.2/setup.py:67: DeprecationWarning: [DEBUG ] [DEBUG ] numpy.distutils is deprecated since NumPy 1.23.0, as a result [DEBUG ] of the deprecation of distutils itself. It will be removed for [DEBUG ] Python >= 3.12. For older Python versions it will remain present. [DEBUG ] It is recommended to use setuptools < 60.0 for those Python versions. [DEBUG ] For more details, see: [DEBUG ] https://numpy.org/devdocs/reference/distutils_status_migration.html [DEBUG ] [DEBUG ] [DEBUG ] import numpy.distutils.command.sdist [DEBUG ] Cythonizing sources [DEBUG ] Processing numpy/random/_bounded_integers.pxd.in [DEBUG ] Processing numpy/random/_philox.pyx [DEBUG ] warning: _philox.pyx:14:0: The 'DEF' statement is deprecated and will be removed in a future Cython version. Consider using global variables, constants, and in-place literals instead. See https://github.com/cython/cython/issues/4310 [DEBUG ] [DEBUG ] Error compiling Cython file: [DEBUG ] ------------------------------------------------------------ [DEBUG ] ... [DEBUG ] self.rng_state.ctr.v[i] = counter[i] [DEBUG ] [DEBUG ] self._reset_state_variables() [DEBUG ] [DEBUG ] self._bitgen.state = <void >&self.rng_state [DEBUG ] self._bitgen.next_uint64 = &philox_uint64 [DEBUG ] ^ [DEBUG ] ------------------------------------------------------------ [DEBUG ] [DEBUG ] _philox.pyx:190:35: Cannot assign type 'uint64_t ()(void ) except? -1 nogil' to 'uint64_t ()(void ) noexcept nogil' [DEBUG ] Traceback (most recent call last): [DEBUG ] File "/Users/fprefect/Developer/kivy/build/numpy/x86_64/numpy-1.24.2/tools/cythonize.py", line 205, in [DEBUG ] main() [DEBUG ] File "/Users/fprefect/Developer/kivy/build/numpy/x86_64/numpy-1.24.2/tools/cythonize.py", line 201, in main [DEBUG ] find_process_files(root_dir) [DEBUG ] File "/Users/fprefect/Developer/kivy/build/numpy/x86_64/numpy-1.24.2/tools/cythonize.py", line 192, in find_process_files [DEBUG ] process(root_dir, fromfile, tofile, function, hash_db) [DEBUG ] File "/Users/fprefect/Developer/kivy/build/numpy/x86_64/numpy-1.24.2/tools/cythonize.py", line 158, in process [DEBUG ] processor_function(fromfile, tofile) [DEBUG ] File "/Users/fprefect/Developer/kivy/build/numpy/x86_64/numpy-1.24.2/tools/cythonize.py", line 47, in process_pyx [DEBUG ] subprocess.check_call( [DEBUG ] File "/Users/fprefect/Developer/kivy/dist/hostpython3/lib/python3.10/subprocess.py", line 369, in check_call [DEBUG ] raise CalledProcessError(retcode, cmd) [DEBUG ] subprocess.CalledProcessError: Command '['/Users/fprefect/Developer/kivy/dist/hostpython3/bin/python', '-m', 'cython', '-3', '--fast-fail', '-o', '_philox.c', '_philox.pyx']' returned non-zero exit status 1. [DEBUG ] Traceback (most recent call last): [DEBUG ] File "/Users/fprefect/Developer/kivy/build/numpy/x86_64/numpy-1.24.2/setup.py", line 479, in [DEBUG ] setup_package() [DEBUG ] File "/Users/fprefect/Developer/kivy/build/numpy/x86_64/numpy-1.24.2/setup.py", line 458, in setup_package [DEBUG ] generate_cython() [DEBUG ] File "/Users/fprefect/Developer/kivy/build/numpy/x86_64/numpy-1.24.2/setup.py", line 261, in generate_cython [DEBUG ] raise RuntimeError("Running cythonize failed!") [DEBUG ] RuntimeError: Running cythonize failed! Exception in thread background thread for pid 73573: Traceback (most recent call last): File "/Users/fprefect/.pyenv/versions/3.10.10/lib/python3.10/threading.py", line 1016, in _bootstrap_inner self.run() File "/Users/fprefect/.pyenv/versions/3.10.10/lib/python3.10/threading.py", line 953, in run self._target(self._args, *self._kwargs) File "/Users/fprefect/Developer/kivy/venv/lib/python3.10/site-packages/sh.py", line 1655, in wrap fn(rgs, **kwargs) File "/Users/fprefect/Developer/kivy/venv/lib/python3.10/site-packages/sh.py", line 2657, in background_thread handle_exit_code(exit_code) File "/Users/fprefect/Developer/kivy/venv/lib/python3.10/site-packages/sh.py", line 2348, in fn return self.command.handle_command_exit_code(exit_code) File "/Users/fprefect/Developer/kivy/venv/lib/python3.10/site-packages/sh.py", line 839, in handle_command_exit_code raise exc sh.ErrorReturnCode_1:

RAN: /Users/fprefect/Developer/kivy/dist/hostpython3/bin/python setup.py build_ext -g

STDOUT: Running from numpy source directory. /Users/fprefect/Developer/kivy/build/numpy/x86_64/numpy-1.24.2/setup.py:67: DeprecationWarning:

numpy.distutils is deprecated since NumPy 1.23.0, as a result of the deprecation of distutils itself. It will be removed for Python >= 3.12. For older Python versions it will remain present. It is recommended to use setuptools < 60.0 for those Python versions. For more details, see: https://numpy.org/devdocs/reference/distutils_status_migration.html

import numpy.distutils.command.sdist Cythonizing sources Processing numpy/random/_bounded_integers.pxd.in Processing numpy/random/_philox.pyx warning: _philox.pyx:14:0: The 'DEF' statement is deprecated and will be removed in a future Cyt... (2281 more, please see e.stdout)

STDERR:

Traceback (most recent call last): File "/Users/fprefect/Developer/kivy/venv/bin/toolchain", line 8, in sys.exit(main()) File "/Users/fprefect/Developer/kivy/venv/lib/python3.10/site-packages/kivy_ios/toolchain.py", line 1564, in main ToolchainCL() File "/Users/fprefect/Developer/kivy/venv/lib/python3.10/site-packages/kivy_ios/toolchain.py", line 1308, in init getattr(self, args.command)() File "/Users/fprefect/Developer/kivy/venv/lib/python3.10/site-packages/kivy_ios/toolchain.py", line 1377, in build build_recipes(args.recipe, ctx) File "/Users/fprefect/Developer/kivy/venv/lib/python3.10/site-packages/kivy_ios/toolchain.py", line 1142, in build_recipes recipe.execute() File "/Users/fprefect/Developer/kivy/venv/lib/python3.10/site-packages/kivy_ios/toolchain.py", line 704, in execute self.build_all() File "/Users/fprefect/Developer/kivy/venv/lib/python3.10/site-packages/kivy_ios/toolchain.py", line 74, in _cache_execution f(self, *args, *kwargs) File "/Users/fprefect/Developer/kivy/venv/lib/python3.10/site-packages/kivy_ios/toolchain.py", line 803, in build_all self.build(arch) File "/Users/fprefect/Developer/kivy/venv/lib/python3.10/site-packages/kivy_ios/toolchain.py", line 74, in _cache_execution f(self, args, **kwargs) File "/Users/fprefect/Developer/kivy/venv/lib/python3.10/site-packages/kivy_ios/toolchain.py", line 790, in build self.build_arch(arch) File "/Users/fprefect/Developer/kivy/venv/lib/python3.10/site-packages/kivy_ios/recipes/numpy/init.py", line 36, in build_arch super().build_arch(arch) File "/Users/fprefect/Developer/kivy/venv/lib/python3.10/site-packages/kivy_ios/toolchain.py", line 1095, in build_arch shprint(hostpython, "setup.py", "build_ext", "-g", File "/Users/fprefect/Developer/kivy/venv/lib/python3.10/site-packages/kivy_ios/toolchain.py", line 56, in shprint for line in cmd: File "/Users/fprefect/Developer/kivy/venv/lib/python3.10/site-packages/sh.py", line 895, in next self.wait() File "/Users/fprefect/Developer/kivy/venv/lib/python3.10/site-packages/sh.py", line 812, in wait self.handle_command_exit_code(exit_code) File "/Users/fprefect/Developer/kivy/venv/lib/python3.10/site-packages/sh.py", line 839, in handle_command_exit_code raise exc sh.ErrorReturnCode_1:

RAN: /Users/fprefect/Developer/kivy/dist/hostpython3/bin/python setup.py build_ext -g

STDOUT: Running from numpy source directory. /Users/fprefect/Developer/kivy/build/numpy/x86_64/numpy-1.24.2/setup.py:67: DeprecationWarning:

numpy.distutils is deprecated since NumPy 1.23.0, as a result of the deprecation of distutils itself. It will be removed for Python >= 3.12. For older Python versions it will remain present. It is recommended to use setuptools < 60.0 for those Python versions. For more details, see: https://numpy.org/devdocs/reference/distutils_status_migration.html

import numpy.distutils.command.sdist Cythonizing sources Processing numpy/random/_bounded_integers.pxd.in Processing numpy/random/_philox.pyx warning: _philox.pyx:14:0: The 'DEF' statement is deprecated and will be removed in a future Cyt... (2281 more, please see e.stdout)

STDERR:



**Screenshots**
<!--
ONLY for XCode related errors, use the LOGS section for toolchain.py related problems!
 If applicable, add screenshots to help explain your problem.
-->

**Additional context**
Add any other context about the problem here.
Tan837 commented 11 months ago

Many people are facing this issue. This is a problem from the Kivy developer's end. They have changed something. I hope they fix it soon.

pythonsus commented 11 months ago

use this https://github.com/jswillard/kivy-ios/tree/master

pythonsus commented 11 months ago

it is a cython version problem since a new version came out yesterday which is 3.0.0

misl6 commented 10 months ago

Fixed via #844