JIT & macOS fat builds

[ronaldoussoren]

We support fat builds of Python on macOS, in particular used for the "Universal 2" installers on macOS (configure --enable-universalsdk --with-universal-archs=universal2). This uses clang's builtin support for compiling multiple architectures in one go (e.g. clang -arch arm64 -arch x86_64 ...).

As far as I can see now the machinery for JIT (--enable-experimental-jit) does not support this at the moment.

On possible way to get this to work (untested and likely incomplete) is to generate multiple variants for jit_stencils.h when doing a fat build and generate an umbrella header that includes the right one:

// jit_stencils.h

#if defined(__arm64__)
#include "jit_stencils-arm64.h"
#elif defined(__x86_64__)
#include "jit_stencils-x86_64.h"
#endif

Linked PRs

• gh-115742

[brandtbucher]

I agree that we should support this.

Something that should be fixed (or figured out) first is getting --with-universal-archs with just one platform to work correctly. For example, if I'm on an M2 Mac and ./configure --enable-universalsdk --with-universal-archs=intel-64, the host configure var is set to aarch64-apple-darwin23.2.0 (it even prints out a message that I'm building for a tier two platform).

host is what's used to build the JIT, so mayhem ensues. Maybe we should use a new var, that can hold several space-separated values?

Once we have cross-builds on one platform working on macOS, it probably wouldn't be too hard to build two (or more!) JITs from there by passing several platforms to the build script.

I'm happy to take on the JIT side of this project. Can somebody else figure out the prerequisite configure stuff? :)

[ronaldoussoren]

The patch below teaches configure.ac about mixing --experimental-jit and --with-universal-archs. The Tools/jit/build.py script will get called with multiple target triples (except for intel-64 and intel-32 which describe only 1 architecture).

I had to move handling --with-universal-archs to below handling --with-universal-archs, other than that the change it fairly mundane. Setting ARCH_TRIPPLES is a bit weird, mostly done this way to save some typing.

BTW. The patch can result in calling the JIT tooling with a number of ancient Mac architectures (like PPC and i386), feel free to error out on those. IMHO we should remove those from configure entirely, but that's for a different issue.

diff --git a/configure.ac b/configure.ac
index b39af7422c..9f560e6088 100644
--- a/configure.ac
+++ b/configure.ac
@@ -1621,27 +1621,6 @@ else
   AC_MSG_RESULT([no])
 fi

-# Check for --enable-experimental-jit:
-AC_MSG_CHECKING([for --enable-experimental-jit])
-AC_ARG_ENABLE([experimental-jit],
-              [AS_HELP_STRING([--enable-experimental-jit],
-                              [build the experimental just-in-time compiler (default is no)])],
-              [],
-              [enable_experimental_jit=no])
-AS_VAR_IF([enable_experimental_jit],
-          [no],
-          [],
-          [AS_VAR_APPEND([CFLAGS_NODIST], [" -D_Py_JIT"])
-           AS_VAR_SET([REGEN_JIT_COMMAND],
-                      ["\$(PYTHON_FOR_REGEN) \$(srcdir)/Tools/jit/build.py $host"])
-           AS_VAR_SET([JIT_STENCILS_H], ["jit_stencils.h"])
-           AS_VAR_IF([Py_DEBUG],
-                     [true],
-                     [AS_VAR_APPEND([REGEN_JIT_COMMAND], [" --debug"])],
-                     [])])
-AC_SUBST([REGEN_JIT_COMMAND])
-AC_SUBST([JIT_STENCILS_H])
-AC_MSG_RESULT([$enable_experimental_jit])

 # Enable optimization flags
 AC_SUBST([DEF_MAKE_ALL_RULE])
@@ -2436,42 +2415,50 @@ yes)
                UNIVERSAL_ARCH_FLAGS="-arch ppc -arch i386"
                LIPO_32BIT_FLAGS=""
                ARCH_RUN_32BIT=""
+              ARCH_TRIPPLES=`echo {ppc,i386}-apple-darwin`
                ;;
             64-bit)
                UNIVERSAL_ARCH_FLAGS="-arch ppc64 -arch x86_64"
                LIPO_32BIT_FLAGS=""
                ARCH_RUN_32BIT="true"
+              ARCH_TRIPPLES=`echo {ppc64,x86_64}-apple-darwin`
                ;;
             all)
                UNIVERSAL_ARCH_FLAGS="-arch i386 -arch ppc -arch ppc64 -arch x86_64"
                LIPO_32BIT_FLAGS="-extract ppc7400 -extract i386"
                ARCH_RUN_32BIT="/usr/bin/arch -i386 -ppc"
+              ARCH_TRIPPLES=`echo {i386,ppc,ppc64,x86_64}-apple-darwin`
                ;;
             universal2)
                UNIVERSAL_ARCH_FLAGS="-arch arm64 -arch x86_64"
                LIPO_32BIT_FLAGS=""
                LIPO_INTEL64_FLAGS="-extract x86_64"
                ARCH_RUN_32BIT="true"
+              ARCH_TRIPPLES=`echo {arm64,x86_64}-apple-darwin`
                ;;
             intel)
                UNIVERSAL_ARCH_FLAGS="-arch i386 -arch x86_64"
                LIPO_32BIT_FLAGS="-extract i386"
                ARCH_RUN_32BIT="/usr/bin/arch -i386"
+              ARCH_TRIPPLES=`echo {i386,x86_64}-apple-darwin`
                ;;
             intel-32)
                UNIVERSAL_ARCH_FLAGS="-arch i386"
                LIPO_32BIT_FLAGS=""
                ARCH_RUN_32BIT=""
+              ARCH_TRIPPLES=i386-apple-darwin
                ;;
             intel-64)
                UNIVERSAL_ARCH_FLAGS="-arch x86_64"
                LIPO_32BIT_FLAGS=""
                ARCH_RUN_32BIT="true"
+              ARCH_TRIPPLES=x86_64-apple-darwin
                ;;
             3-way)
                UNIVERSAL_ARCH_FLAGS="-arch i386 -arch ppc -arch x86_64"
                LIPO_32BIT_FLAGS="-extract ppc7400 -extract i386"
                ARCH_RUN_32BIT="/usr/bin/arch -i386 -ppc"
+              ARCH_TRIPPLES=`echo {i386,ppc,x86_64}-apple-darwin`
                ;;
             *)
                AC_MSG_ERROR([proper usage is --with-universal-arch=universal2|32-bit|64-bit|all|intel|3-way])
@@ -2565,6 +2552,28 @@ yes)
     ;;
 esac

+# Check for --enable-experimental-jit:
+AC_MSG_CHECKING([for --enable-experimental-jit])
+AC_ARG_ENABLE([experimental-jit],
+              [AS_HELP_STRING([--enable-experimental-jit],
+                              [build the experimental just-in-time compiler (default is no)])],
+              [],
+              [enable_experimental_jit=no])
+AS_VAR_IF([enable_experimental_jit],
+          [no],
+          [],
+          [AS_VAR_APPEND([CFLAGS_NODIST], [" -D_Py_JIT"])
+           AS_VAR_SET([REGEN_JIT_COMMAND],
+                      ["\$(PYTHON_FOR_REGEN) \$(srcdir)/Tools/jit/build.py ${ARCH_TRIPPLES:-$host}"])
+           AS_VAR_SET([JIT_STENCILS_H], ["jit_stencils.h"])
+           AS_VAR_IF([Py_DEBUG],
+                     [true],
+                     [AS_VAR_APPEND([REGEN_JIT_COMMAND], [" --debug"])],
+                     [])])
+AC_SUBST([REGEN_JIT_COMMAND])
+AC_SUBST([JIT_STENCILS_H])
+AC_MSG_RESULT([$enable_experimental_jit])
+
 case "$CC_BASENAME" in
 *mpicc*)
     CFLAGS_NODIST="$CFLAGS_NODIST"

[ronaldoussoren]

@brandtbucher, I've added a draft PR that seems to work for me (that is, a universal2 build works and won't crash for both light interactive use and a partial test run for both arm64 and x86_64).

The PR is a bit rough and contains one unrelated change due using the compiler included in Xcode (not command line tools) and installing llvm-16 manually and not through homebrew.

[savannahostrowski]

I was digging into this a bit but wanted to ask a perhaps naive question unrelated to JIT-specific builds: is there a reason we have FAT builds for Mac but not Windows?

I can see the benefits of a universal installer from an end-user perspective, as some users might not know which architecture they're using. However, universal installers also have downsides, such as increased binary size.

Has there been any discussion about providing separate installers for Mac x86 and ARM architectures?