search

beeware / Python-Apple-support

A meta-package for building a version of Python that can be embedded into a macOS, iOS, tvOS or watchOS project.
MIT License
1.11k stars 160 forks source link

Building from source for iOS fails on Big Sur/iOS 14 #106

Closed pisajew closed 3 years ago

pisajew commented 3 years ago

Hi,

I wonder if you could provide some details on required build environment to build an embeddable Python for iOS from source.

My current build environment is:

I start by checking out the dev branch and try to build using make clean iOS

This fails at some point with error:

    ssize_t pwritev(int, const struct iovec *, int, off_t) __DARWIN_NOCANCEL(pwritev) __API_AVAILABLE(macos(11.0), ios(14.0), watchos(7.0), tvos(14.0));
            ^
    ./Modules/posixmodule.c:9689:18: note: enclose 'pwritev' in a __builtin_available check to silence this warning
            result = pwritev(fd, iov, cnt, offset);
                     ^~~~~~~
    6 warnings and 1 error generated.

To fix this I've changed Makefile to set MACOSX_DEPLOYMENT_TARGET to 11 instead of 10.8.

Then attempts to build fail with:

  gcc -Wno-unused-result -Wsign-compare -Wunreachable-code -DNDEBUG -g -fwrapv -O3 -Wall    -Wextra -Wno-unused-result -Wno-unused-parameter -Wno-missing-field-initializers -Wstrict-prototypes -Werror=implicit-function-declaration  -I./Include/internal  -I. -I./Include    -DPy_BUILD_CORE_BUILTIN  -DPy_BUILD_CORE_BUILTIN -I./Include/internal -c ./Modules/posixmodule.c -o Modules/posixmodule.o
./Modules/posixmodule.c:9257:15: error: implicit declaration of function 'sendfile' is invalid in C99 [-Werror,-Wimplicit-function-declaration]
        ret = sendfile(in, out, offset, &sbytes, &sf, flags);
              ^
1 error generated.

To resolve this I apply this patch:

diff --git a/patch/Python/Python.patch b/patch/Python/Python.patch
index d8ab8a3..e8a6b29 100644
--- a/patch/Python/Python.patch
+++ b/patch/Python/Python.patch
@@ -32030,7 +32030,7 @@ index eb0b56aebb..10d2aa2755 100644
  #endif /* __APPLE__ */

  #define PY_SSIZE_T_CLEAN
-@@ -216,6 +218,25 @@
+@@ -216,6 +218,26 @@
  #endif  /* _MSC_VER */
  #endif  /* ! __WATCOMC__ || __QNX__ */

@@ -32053,6 +32053,7 @@ index eb0b56aebb..10d2aa2755 100644
 +#  undef HAVE_WAIT4
 +#  undef HAVE_WAITPID
 +#endif
++#undef HAVE_SENDFILE

  /*[clinic input]
  # one of the few times we lie about this name!

this moves the compilation process a bit further, but eventually fails with:

running build_ext
Traceback (most recent call last):
  File "./setup.py", line 2434, in <module>
    main()
  File "./setup.py", line 2404, in main
    setup(# PyPI Metadata (PEP 301)
  File "/Users/pki/Projekty/python_sandbox/Python-Apple-support/build/macOS/Pyth
on-3.8.3-macosx.x86_64/Lib/distutils/core.py", line 148, in setup
    dist.run_commands()
  File "/Users/pki/Projekty/python_sandbox/Python-Apple-support/build/macOS/Pyth
on-3.8.3-macosx.x86_64/Lib/distutils/dist.py", line 966, in run_commands
    self.run_command(cmd)
  File "/Users/pki/Projekty/python_sandbox/Python-Apple-support/build/macOS/Pyth
on-3.8.3-macosx.x86_64/Lib/distutils/dist.py", line 985, in run_command
    cmd_obj.run()
  File "/Users/pki/Projekty/python_sandbox/Python-Apple-support/build/macOS/Python-3.8.3-macosx.x86_64/Lib/distutils/command/build.py", line 135, in run
    self.run_command(cmd_name)
  File "/Users/pki/Projekty/python_sandbox/Python-Apple-support/build/macOS/Python-3.8.3-macosx.x86_64/Lib/distutils/cmd.py", line 313, in run_command
    self.distribution.run_command(command)
  File "/Users/pki/Projekty/python_sandbox/Python-Apple-support/build/macOS/Python-3.8.3-macosx.x86_64/Lib/distutils/dist.py", line 985, in run_command
    cmd_obj.run()
  File "/Users/pki/Projekty/python_sandbox/Python-Apple-support/build/macOS/Python-3.8.3-macosx.x86_64/Lib/distutils/command/build_ext.py", line 340, in run
    self.build_extensions()
  File "./setup.py", line 320, in build_extensions
    self.detect_modules()
  File "./setup.py", line 1695, in detect_modules
    self.detect_readline_curses()
  File "./setup.py", line 930, in detect_readline_curses
    (tuple(int(n) for n in dep_target.split('.')[0:2])
AttributeError: 'int' object has no attribute 'split'
make[1]: *** [sharedmods] Error 1
make: *** [build/macOS/Python-3.8.3-macosx.x86_64/dist/lib/libpython3.8.a] Error 2

Would it be possible for you to provide some recommendation on the build environment and flow to succesfully build a package for Python 3.8?

freakboy3742 commented 3 years ago

My build environment (and the environment used in CI to generate new releases) is Catalina; I haven't tried using Big Sur. I'm guessing the issues you've found relate to that version bump - we've seen analogous failures in the past. The problem is caused because Apple introduces or deprecates an API that autoconf didn't previously pick up. We'll need to update the patches to accomodate those changes.

Beyond that, the workflow should be nothing more than "make iOS". If you've got homebrew installed and you have gettext installed in that environment, you can get some problems at runtime. There's also a known problem with the webbrowser module at runtime with Python 3.7+; addressing this is on my todo list. Other than those problems, the build should be straightforward - and the Github Actions CI configuration on this repository has been used to produce the most recently published support packages.

pisajew commented 3 years ago

Thanks. It was the gettext issue, as you described. Uninstalling it via homebrew fixes the issue for me. As a side effect I can confirm that dev builds on BigSur. It seems however that it needs the HAVE_SENDFILE undef patch.