Build Apple Silicon wheels

[messense]

packaging workflow run example: https://github.com/messense/blake3-py/actions/runs/1188683206

Closes #20

[oconnor663]

Thank you! This will save me a lot of trouble. In other projects (https://github.com/oconnor663/blake3-py/blob/master/.github/workflows/upload_github_release_asset.py#L14-L19) I've tried to put together a "rerelease" mechanism, so that I can release new builds like this without tagging a totally new project version. I might try something similar here. But let me know if there are better ways to do that...

[messense]

I think you can try to automate something like that on GitHub Actions with the workflow_dispatch trigger.

I'd edit the tag.yml and add workflow_dispatch trigger like this

name: packaging

on:
  push:
    tags:
      - "*"
    workflow_dispatch:
      inputs:
        tag:
          description: Git tag to update releases

jobs:
  wheel:
    steps:
      - name: upload release asset
        env:
          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
          GITHUB_TAG: ${{ github.event.inputs.tag || github.ref }}
        run: python -u .github/workflows/upload_github_release_asset.py ${{ steps.build_wheel.outputs.wheel_path }}

[oconnor663]

I'm seeing some build failures on these new targets, and coming back to this, it's making me wonder whether some of these builds might not have been doing what we thought they were doing. For example, I don't think we ever pass --target to either cargo build or maturin build. I might've been building e.g. the wrong Windows targets without noticing. I need to look at this much more closely.

I expect that once I fix things on x86_64, the target in this PR is going to stop working. For example, I don't think we specify PYO3_CROSS_INCLUDE_DIR or PYO3_CROSS_LIB_DIR, which according to the PyO3 docs are necessary for cross compilation. I might need to revert this PR. Will post more here when I figure out what's up.

[oconnor663]

(Oh actually we do set CARGO_BUILD_TARGET. I'm slowing remembering this stuff.)

[oconnor663]

Here are some of the build failures I'm seeing: https://github.com/oconnor663/blake3-py/runs/3969628042?check_suite_focus=true

[oconnor663]

In file included from c/blake3_neon.c:1:
In file included from c/blake3_impl.h:4:
In file included from /Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk/usr/include/assert.h:42:
/Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk/usr/include/sys/cdefs.h:807:2: error: Unsupported architecture
#error Unsupported architecture
 ^
In file included from c/blake3_neon.c:1:
In file included from c/blake3_impl.h:7:
In file included from /Applications/Xcode_12.4.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/lib/clang/12.0.0/include/stdint.h:52:
In file included from /Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk/usr/include/stdint.h:52:
In file included from /Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk/usr/include/sys/_types.h:33:
/Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk/usr/include/machine/_types.h:34:2: error: architecture not supported
#error architecture not supported
 ^
In file included from c/blake3_neon.c:1:
In file included from c/blake3_impl.h:7:
In file included from /Applications/Xcode_12.4.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/lib/clang/12.0.0/include/stdint.h:52:
In file included from /Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk/usr/include/stdint.h:52:
/Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk/usr/include/sys/_types.h:55:9: error: unknown type name '__int64_t'
typedef __int64_t       __darwin_blkcnt_t;      /* total blocks */

[oconnor663]

Sorry for all the comment spam. The immediate cause of these is probably that blake3 v1.1.0 enabled the NEON implementation by default, but I'm not totally sure.

[messense]

I think it should be fixable by switching to macos-11 runners.

[messense]

Opened https://github.com/oconnor663/blake3-py/pull/23

[oconnor663]

Ok! The just-released version 0.2.1 includes the new binary wheels for Apple silicon. Do you have a machine to test them out? :) Thanks again for all your help with this.

[messense]

It works on my M1 macbook pro.

❯ uname -a
Darwin messenses-MacBook-Pro.local 20.6.0 Darwin Kernel Version 20.6.0: Mon Aug 30 06:12:20 PDT 2021; root:xnu-7195.141.6~3/RELEASE_ARM64_T8101 arm64

~
❯ pyenv activate test

~ via 🐍 v3.9.2 (test)
❯ pip --version
pip 21.3 from /Users/messense/.pyenv/versions/3.9.2/envs/test/lib/python3.9/site-packages/pip (python 3.9)

~ via 🐍 v3.9.2 (test)
❯ pip install blake3
Collecting blake3
  Using cached blake3-0.2.1-cp39-cp39-macosx_11_0_arm64.whl (278 kB)
Installing collected packages: blake3
Successfully installed blake3-0.2.1
WARNING: You are using pip version 21.3; however, version 21.3.1 is available.
You should consider upgrading via the '/Users/messense/.pyenv/versions/3.9.2/envs/test/bin/python3.9 -m pip install --upgrade pip' command.

~ via 🐍 v3.9.2 (test) took 3s
❯ ipython
Python 3.9.2 (default, Mar 20 2021, 21:34:09)
Type 'copyright', 'credits' or 'license' for more information
IPython 7.21.0 -- An enhanced Interactive Python. Type '?' for help.

In [1]: from blake3 import blake3

In [2]: hash1 = blake3(b"foobarbaz").digest()

In [3]: print(hash1)
b'\xc0\x9a\xfe\xe0\xc9\xf3a\xfba\xe5\xff(\xa7s\x98\x93\xdevo\xb4p\xc5\xfa\x82\xb4\xe5\xe3\x1d\xe2\x7f\xba\xd4'

In [4]: exit

~ via 🐍 v3.9.2 (test) took 2m49s

❯ file /Users/messense/.pyenv/versions/3.9.2/envs/test/lib/python3.9/site-packages/blake3/blake3.cpython-39-darwin.so
/Users/messense/.pyenv/versions/3.9.2/envs/test/lib/python3.9/site-packages/blake3/blake3.cpython-39-darwin.so: Mach-O 64-bit dynamically linked shared library arm64