Apple Silicon Support?

[efroemling]

Now that arm Macs are a thing, just curious if you have plans for universal binary support in the Mac builds?.. (or should this already work?). It seems Python officially supports them with 3.9.1+ but homebrew has older versions building on arm too.

[freakboy3742]

Supporting Apple Silicon hardware is absolutely on to the TODO list. Python 3.9 support has been slow in arriving specifically because of the changes that were merged to support ARM architectures; the official patches clash with the historical patches that I've applied to support Python on iOS.

The biggest issue is getting access hardware to test & build on. I don't currently have access to an Apple Silicon Mac, and I don't know what Github's plans for rolling out Apple Silicon hardware in their build cluster.

I'm also unclear how PyPI handles binary modules. That shouldn't affect this support package too much, but it will affect the general packaging story.

In the meantime, if anyone who has access to Apple Silicon hardware wants to contribute patches, we'd be more than happy to consider them. Or... if someone wants to buy me a new Mac... :-)

(Jokes aside: if someone is willing to fund the work, that work will get prioritised; if you are interested in providing funding, get in touch)

[samueljohn]

@freakboy3742 well, can we talk? I think we can perhaps sponsor you a personal M1 mac.

[freakboy3742]

A status update for those watching this ticket - thanks to generous sponsorship from @samueljohn, we've now obtained an Apple Silicon M1 Mac mini, and we're investigating what we need to do to support this new architecture.

[freakboy3742]

I've just pushed an update to the dev branch that modifies the build to generate a Universal2 binary on macOS. In my initial testing, this means Briefcase can now generate M1-compatible binaries.

The best part is that you don't need to have an M1 Mac to produce an M1-compatible binary. The support package produced by the dev branch can be built on intel hardware, and used on either M1 or Intel hardware, producing apps that are compatible with both M1 and Intel.

I haven't tagged this for a formal release yet - as there are three notable caveats at this time:

1. This update will only work for Python 3.9. A backport to Python 3.8 and earlier is possible in principle, but will be non-trivial
2. I haven't had a chance to test binary module support. Pip 21.0 included support for universal2 wheels; however, I haven't been able to find an example of a package that is packaged using this tag. As soon as I do (I might end up resorting to building my own wheel for a small binary module), I will be able to clarify the state of play with wheels.
3. Big Sur (which is the only macOS version that supports M1 at present) also imposes some stronger code signing requirements. There's a branch in progress that would alter the way Briefcase packages macOS binaries that will be more compatible with Big Sur's code signing requirements.

[freakboy3742]

Ok - so, universal2 support is evidently a work in progress for wheels.

1. It's possible to build an arm64 wheel if you're on M1 hardware.
2. It's possible to build an x86_64 wheel if you're on x86_64 hardware.
3. Pip contains support for universal2 wheels
4. There's a known bug with Universal2 wheel support.

So - there's not much we can't do from BeeWare's side - this is something that will need to be addressed upstream.

[freakboy3742]

Another update:

beeware/briefcase#525 contains a candidate for support of Apple Silicon on macOS. If you want to test this branch, you'll need to point your pyproject.toml at some in-development templates and support package builds - details in the PR (or ask on Gitter if you want more clarification). This release also cleans up the way that apps report in the system monitor, and makes the app fully compatible with Apple's signing/notarizization processes.

We've also added an Xcode output format, in case you want to have direct control over how your app binary is built.

Support for binary packages remains an issue; this is something that the Python ecosystem as a whole will need to come to terms with.

[freakboy3742]

I've just released Briefcase 0.3.5 (as well as related support packages and template updates) that adds support for Apple Silicon on Python 3.9.

The issues with packaging binary modules remain; however they're really an "upstream" problem - as soon as Python/pip/pypi has a full universal2 binary module story, Briefcase (should) pick that up that support automatically.

On that basis, I'm going to close this ticket.

Huge thanks to:

• @samueljohn and @OtoJig for funding some M1 so that we could develop and test Apple Silicon support
• @ButterflyNetwork who funded the Python 3.9 port, which was a vital precursor to this work.
• @SamSchott for his work on beeware/briefcase#525, which changed the way macOS apps are packaged.

[samschott]

Looking at https://bugs.python.org/issue42619, it appears that universal2 wheels and .so files can indeed by built on macOS 11. I've also manually confirmed this on an Intel Mac. The support package at the moment reflects the situation as described in the referenced bug, the Python executable itself is universal but the .so files are not. Nevertheless, the cookiecutter templates claim to produce universal apps which can run natively. This leads to nasty surprises when users actually try to run an app on a M1 Mac without manually forcing Rosetta translation.

@freakboy3742 , would you consider either of the following:

1. Rebuild the support package on macOS 11 (which is provided as a preview on GitHub Actions).
2. Update the app templates to prefer running under Rosetta, for instance by specifying a preferred architecture in the plist file with LSArchitecturePriority.

Regarding 3rd party packages: as you say, pip supports universal2 wheels and they can be built on macOS 11. Most packages on PyPI don't provide universal2 wheels yet but I expect this to change in due course. For now, people can resort to building them themselves if needed...

[freakboy3742]

Thanks for the report @SamSchott. Happy to rebuild the support packages; that looks like the easiest (and long term) option here. That also gives an opportunity to downgrade the 3.9 package to 3.9.1 until we can get to the bottom of the segfault on iOS.

[samschott]

Sounds good to me:) Apart from the support package issue, creating a universal app bundle was really just as simple as tweaking briefcase to build all Python packages from source when installing them into the bundle. And of course having a Mac or CI with macOS 11.

[freakboy3742]

@SamSchott FYI: I've just published Python-Apple-support 3.9-b3, using macOS 11 as a build platform.