Mac arm64 build for 7.7.x

[bernhard-42]

According to https://github.com/CadQuery/cadquery/issues/1236#issuecomment-1401104843 OCP now can build arm64 libs for Mac as conda packages. Will Mac arm64 be supported by the upcoming 7.7.x wheel too?

[fpq473]

In theory we could just add it here:

https://github.com/CadQuery/ocp-build-system/blob/da4c4dd695874297d9327fde6c8de27115f56f1d/.github/workflows/build-wheel.yml#L18

but are github actions available on macos/arm64? A cursory search suggests no.

[jmwright]

I saw the other day that support for GitHub-hosted Apple Silicon is in beta and is expected to go public in Q3 2023: https://github.com/github/roadmap/issues/528

They have that conversation locked, probably because so many devs like me would bug them about it when it will be released. They've supported self-hosted M1 runners since August 2022.

The OCP wheel build should support the GitHub-hosted M1 runner when it's available, but we either have to wait until Q3 2023, or somebody has to set up a self-hosted runner.

[bernhard-42]

Hmmm, wasn't aware of this dependency. Thanks for looking into it!

[fpq473]

@fungiboletus notes in https://github.com/CadQuery/OCP/issues/108#issuecomment-1513043129:

A workaround could be about using docker buildx with qemu. It works relatively well in github actions, though you have to build things inside a container and not directly in the action.

[akoen]

https://github.blog/changelog/2023-10-02-github-actions-apple-silicon-m1-macos-runners-are-now-available-in-public-beta/

[jmwright]

Thanks for the heads-up @akoen

[jmwright]

I tried to enable this runner, unfortunately GitHub does not have a free version for open source projects yet (and I don't know if they have plans to). See the comment here.

[bernhard-42]

@jmwright I build the M1 wheels for myself using ocp-build-system (and host them in my own pypi server locally)

# size       wheel
  161081972  cadquery_ocp-7.7.1-cp310-cp310-macosx_11_0_arm64.whl
  161082182  cadquery_ocp-7.7.1-cp311-cp311-macosx_11_0_arm64.whl
  161087133  cadquery_ocp-7.7.1-cp39-cp39-macosx_11_0_arm64.whl

I could contribute them, but don't know how my wheels could be integrated into your workflow.

[jmwright]

It's not a solution for this issue, but someone posted on the CadQuery repo that they were able to get CadQuery to run on an M1 Mac. https://github.com/CadQuery/cadquery/issues/1434

[jeremiahrose]

There is no ETA for free osx-arm64 runners from Github and they apparently aren't even working on it at present (source). And there are no linux-aarch64 runners or ETA for them being available.

However, if the Conda package is already being cross compiled to arm64, then surely it would be possible to convert that recipe to Pip and use it in this repo on an intel runner - right?

I'd be happy to assist with this as I really need an easily-installable-with-pip Cadquery for my team.

[jeremiahrose]

Here is the PR that added osx-arm64 to conda: https://github.com/conda-forge/ocp-feedstock/pull/17

[jeremiahrose]

After trying a few different approaches, building locally ended up being the path of least resistance. I'd be happy to do this on a regular basis on my M1 Macbook and upload the wheels to PyPi if that'd be helpful?

PR containing the build script here: https://github.com/CadQuery/ocp-build-system/pull/18

[rawwerks]

just curious if there is a plan to take @jeremiahrose up on his offer (https://github.com/CadQuery/ocp-build-system/pull/18), or if i should plan to work around this issue for the foreseeable future.

[jmwright]

I have Mac M1 wheels built, but I can't upload them to PyPI right now because of size limits. I can make them available via something like a Dropbox download if someone has the capability and interest to install them locally.

[jeremiahrose]

Can we request a size limit increase from PyPI? I hear they do regularly grant them on a case by case basis.

In the meantime I have uploaded 7.7.1.0 to GitHub releases on my fork, you can install directly with:

pip3 install https://github.com/jeremiahrose/ocp-build-system/releases/download/7.7.1.0/cadquery_ocp-7.7.1-cp310-cp310-macosx_11_0_arm64.whl

There are wheels for other Python versions as well

[jmwright]

I already requested a size increase but have not heard back. We've gotten one increase already.

[KenjiTakahashi]

FYI: https://www.flyci.net [I'm not affiliated.]

[bernhard-42]

@jmwright Since vtk in the meantime has Mac arm wheels, I wonder if you could remove the vtk libraries from cadquery_ocp. Given the amount of wheels this might free up enough to add another wheel?

[bernhard-42]

Plus I would think about removing the Python 3.8 wheels. 3.9, 3.10, 3.11 and 3.12 should be sufficient in 2024

[jmwright]

@bernhard-42 We started building and bundling VTK ourselves because the official VTK wheels were built against a libstdc++ that was so new only bleeding-edge distros could run them. If that is no longer an issue we can switch back to the official VTK wheels.

Am I reading this article correctly that Python 3.8 goes EoL during 2024?

[bernhard-42]

I would say that build123d and cadquery are more domain specific languages than generic Python packages that one installs into old Python envs. As such, I would expect that supporting the latest 4 Python versions from 3.9 onwards should be really sufficient. For example, the newest numpy and pandas also support >= 3.9 only.

If we look at the hassle every arm mac user has to go through at the moment, upgrading from Python 3.8 to 3.9 shouldn't be a big deal.

But that of course are my 2 cent only.

[KenjiTakahashi]

It's worth noting also that the latest macOS already ships with Python 3.9, so then no need to touch anything even.

[jmwright]

Related comment on the CadQuery repo about removing Python 3.8 support: https://github.com/CadQuery/cadquery/pull/1498#issuecomment-1894508055

[jmwright]

FYI: https://github.com/CadQuery/ocp-build-system/commit/f4c249818cc9ade2c9ec0cdfe74b1e9207c1e175

[bernhard-42]

great! Thank you.

[radeva]

I tried to enable this runner, unfortunately GitHub does not have a free version for open source projects yet .. After trying a few different approaches, building locally ended up being the path of least resistance.

@jmwright @jeremiahrose You can try FlyCI's M1 and M2 runners. They are on average 2x faster and 2x cheaper than GitHub's AND we have a free tier for OSS projects (see below).

Install Instructions

1. Install FlyCI app and
2. Easily replace one line of code and start using FlyCI runners:

jobs:
 ci:
-    runs-on: macos-latest
+    runs-on: flyci-macos-large-latest-m1
   steps:
   - name: 👀 Checkout repo
     uses: actions/checkout@v4

500 mins/month Free for Public Repos

Since your repo is public, FlyCI offers 500 mins/month of free M1 runner usage with the flyci-macos-large-latest-m1 runner for public projects.

Don't hesitate to contact us in case the free tier doesn't suit your needs or you experience any issues with the runners. Our team is here to support you!

Best Regards, Veselina Radeva Product Manager at FlyCI

[jdegenstein]

FYI, Apple Silicon runners are now available for free from GitHub for public repos. Blog post here: https://github.blog/changelog/2024-01-30-github-actions-introducing-the-new-m1-macos-runner-available-to-open-source/

I tested it and it is working great! Oh and thanks to barnaby for letting me know about it!

[jmwright]

Thanks for letting me know @jdegenstein

Do you have an example of how you used it? I thought I followed the instructions in the blog post, but the GitHub Action is still building x86_64 wheels for me.

[jdegenstein]

Sure, here is my test run file https://github.com/jdegenstein/build123d/blob/GH_macARM64_onlyPlane/.github/workflows/test.yml

Definitely looks like arm64 to me:

[jdegenstein]

@jmwright I was able to build cadquery-ocp on my fork for MacOS arm64. Long story short, the biggest issue was that conda was creating an "osx-64" env when what was needed was "osx-arm64". There are a number of other minor issues and cleanup but it should provide a starting point at least:

https://github.com/jdegenstein/ocp-build-system/actions/runs/7747906519

I also published a cp310 wheel here: https://github.com/jdegenstein/ocp-build-system/releases/tag/7.7.2_macos_arm64_cp310

[jmwright]

Thanks @jdegenstein . Would you like to do a PR to the main repo? That workflow file is not in use yet, and was added specifically for this purpose, so I can merge it and then do some cleanup. I may switch all the builds to the local build method eventually, but want to be sure to get MacOS ARM64 builds working first.

[hf-krechan]

Great work @jdegenstein 🚀 Thanks for this setup of the GitHub Action

[jmwright]

Thanks to @jdegenstein there are now MacOS ARM64 wheel builds in CI: https://github.com/CadQuery/ocp-build-system/actions/runs/7874432399

The wheel files will have to be renamed before uploading to PyPI, but that will have to wait until we get the project size increased there.

Please test these wheels out and report back here if you have any problems.

[jdegenstein]

Awesome! It would be great if the newly built MacOS ARM64 wheels could be added to a release here on GH as well (that way they can be e.g. pip installed from a URL).

[kdewald]

Awesome development! I can't wait to try this out!

[jmwright]

@jdegenstein I didn't think about users doing a pip install from a specific release/tag. I will try to assemble that release.

[jmwright]

The MacOS ARM64 wheels have been attached to the release here. It is possible to do a pip install of the wheels on that release by copying the URL for the file that matches your OS and Python version and then doing pip install URL_OF_CORRECT_WHEEL_FILE.

[fferri]

Any idea why that would not work?

❯ python3 --version
Python 3.11.6

❯ ls -l $(which python3)
/opt/homebrew/bin/python3 -> ../Cellar/python@3.11/3.11.6_1/bin/python3

❯ python3 -m pip install https://github.com/CadQuery/ocp-build-system/releases/download/7.7.2.0/cadquery_ocp-7.7.2-cp311-cp311-macosx_11_0_arm64.whl
Collecting cadquery-ocp==7.7.2
  Downloading https://github.com/CadQuery/ocp-build-system/releases/download/7.7.2.0/cadquery_ocp-7.7.2-cp311-cp311-macosx_11_0_arm64.whl (165.4 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 165.4/165.4 MB 6.4 MB/s eta 0:00:00
Installing collected packages: cadquery-ocp
Successfully installed cadquery-ocp-7.7.2

❯ python3
Python 3.11.6 (main, Nov  2 2023, 04:39:43) [Clang 14.0.3 (clang-1403.0.22.14.1)] on darwin
>>> import cadquery
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ModuleNotFoundError: No module named 'cadquery'

❯ ls -ld /opt/homebrew/Cellar/python@3.11/3.11.6_1/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/*cadquery*
/opt/homebrew/Cellar/python@3.11/3.11.6_1/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/cadquery_ocp-7.7.2.dist-info

[jmwright]

You need to install the cadquery package as well. cadquery-ocp does not bundle CadQuery with it. It only has a version of OCP that was intended for use with cadquery. We can't make it simply "ocp" because that would cause a naming conflict on PyPI.

[fferri]

Right, thanks for the clarification.

It works fine on macOS arm64. Thanks for making these packages available!

[infused-kim]

Thank you so much for adding Apple Silicon support. I used the workaround of installing the wheel manually and it's working perfectly.

It looks like pypi increased the storage size to accomodate Apple Silicon builds (https://github.com/pypi/support/issues/3631).

Would it be possible to submit the wheels to pypi?

[jmwright]

Please download the correct wheel for your platform and Python version and do a test installation. Once these have been tested, I will publish them to PyPI. During the installation, cadquery-vtk will be downloaded from PyPI since it is no longer embedded inside the OCP wheel.

https://github.com/CadQuery/ocp-build-system/releases/tag/7.7.2.1

[jmwright]

MacOS ARM64 builds are available for 7.7.2.1 on PyPI now.

https://pypi.org/project/cadquery-ocp/

[bernhard-42]

Thank you.

Unfortunately I can't test it since the arm build of macos is for version 14.0+ and I have 13.0. The intel build is for macos 13+, but I have arm - LOL

[cadquery_vtk-9.2.6-cp311-cp311-macosx_14_0_arm64.whl ](https://files.pythonhosted.org/packages/91/7b/b378be0f03ad1ef366d47e5be96178b50704765e1ad6e5a5abfe25683031/cadquery_vtk-9.2.6-cp311-cp311-macosx_14_0_arm64.whl)(39.1 MB [view hashes](https://pypi.org/project/cadquery-vtk/#copy-hash-modal-46839a0f-4beb-41ed-9462-bba03c08b631))
Uploaded 28. Aug. 2024 CPython 3.11 macOS 14.0+ ARM64

[cadquery_vtk-9.2.6-cp311-cp311-macosx_13_0_x86_64.whl ](https://files.pythonhosted.org/packages/ff/5e/678d3a26e77aa79d43509abe56e25522d163057e6960fe41697c2e2b90aa/cadquery_vtk-9.2.6-cp311-cp311-macosx_13_0_x86_64.whl)(42.8 MB [view hashes](https://pypi.org/project/cadquery-vtk/#copy-hash-modal-5014bb6b-8b87-4a08-8c95-fe965fcc73cd))
Uploaded 28. Aug. 2024 CPython 3.11 macOS 13.0+ x86-64

Python packaging for binary packages is simply a nightmare ...

[jmwright]

Python packaging for binary packages is simply a nightmare ...

I 100% agree.

[jdegenstein]

@jmwright I was able to install cadquery-vtk and it worked perfectly on Win10 / py311. Thank you for the work on this!

I guess I need to investigate if it is possible to build on MacOS14 (arm) for MacOS13 (arm). We are building on MacOS14 to ensure we get arm and not x86_64 (github's approach to this problem seems a little lacking in my opinion, but maybe there is another workaround).

[jmwright]

There was an issue with the VTK dependency that is causing this wheel not to work properly for certain use cases, so I am reopening this issue.