Limit macOS platform tag by compiled deployment target?

[QuLogic]

We were recently notified that our moved-to-Meson nightly macOS arm64 wheels do not install. This is because they have a platform tag with 10_12, but arm64 only existed on macOS 11 and above, so those wheels are never considered. These wheels are built with cibuildwheel using x86_64 and cross-compiled arm64 and MACOSX_DEPLOYMENT_TARGET=10.12. So the 10_12 does make sense from the environment variable point of view.

However, our older wheels built with setuptools use this same environment, but get 11_0 in their platform tag.

Some digging shows that wheel actually looks at all .dylib and .so in the produced wheel, and overrides the platform tag to pick a minimum version that matches the minimum deployment target of the compiled extensions. See also the pypa/wheel#314.

A comment elsewhere indicates that Xcode will not produce a deployment target below 11 for arm64 builds.

So these two things work together in a setuptools-based build to produce a platform tag with 11_0 on Apple Silicon, even though the deployment target environment variable is set lower.

For Matplotlib, we can work around this by splitting the job by arch and setting MACOSX_DEPLOYMENT_TARGET as necessary.

But for meson-python, since it re-implements wheel, the question here is whether it should also implement the check on compiled extensions, and then pick a platform tag that reflects what's in the wheel?

[rgommers]

Thanks for the report @QuLogic. This should be easy to change I think, at least to make it at least 11_0 for arm64 wheels. What surprises me is that it matters - the version number means the lowest allowed macOS version to install on, and any higher OS version is allowed as well. So it seems to me that it's a bug in pip if it silently ignores wheels labelled as 10_xx.

Some digging shows that wheel actually looks at all .dylib and .so in the produced wheel, and overrides the platform tag to pick a minimum version that matches the minimum deployment target of the compiled extensions.

That is a lot more work, with code that looks like it is fragile and has the potential to change over time, since it's fishing magic numbers out of binary files. The better way of doing that is probably to use otool -l on each binary file (see this SO answer). It's a lot of subprocess calls to do that though.

The only way I can think of where this matters is if a user would not set MACOSX_DEPLOYMENT_TARGET but would insert -mmacosx-version-min into CFLAGS/c_args. Can anyone think of other scenarios?

[dnicolodi]

We were recently notified that our moved-to-Meson nightly macOS arm64 wheels do not install. This is because they have a platform tag with 10_12, but arm64 only existed on macOS 11 and above, so those wheels are never considered.

How is this possible? The macOS platform version tag is intended to be interpreted as a minimum supported version, not as an exact match. Does pip contain a check that explicitly forbids macOS wheels with platform tag version less than 11 to be installed on arm64?

I am not sure that setting $MACOSX_DEPLOYMENT_TARGET to something invalid should get special treatment by meson-python. I would have expected the macOS toolchain to error out if the env var is set to something that does not make sense.

What does the macOS toolchain on arm64 do when $MACOSX_DEPLOYMENT_TARGET is set to a version smaller than 11.0? Does it silently upgrade it to 11.0 or does it ignore the environment variable completely?

I definitely don't want to enter the busyness of parsing binaries to extract the compatibility version.

[ksunden]

I can say that installing was rejected on my ARM Mac for the matplotlib wheel. (It installed and appeared to work just changing the file name, though)

It certainly seems like universal wheels will install, but arm only ones require at least 11.0.

[rgommers]

Yes, I double checked and pip just refuses, which is arguably a pip bug, but that ship has sailed a long time ago probably (didn't search their issue tracker to verify):

% pip install ./numpy-1.24.4-cp38-cp38-macosx_11_0_arm64.whl
...
Successfully installed numpy-1.24.4

% pip uninstall numpy
...
Successfully uninstalled numpy-1.24.4

% cp numpy-1.24.4-cp38-cp38-macosx_11_0_arm64.whl numpy-1.24.4-cp38-cp38-macosx_10_12_arm64.whl
% pip install ./numpy-1.24.4-cp38-cp38-macosx_10_12_arm64.whl
ERROR: numpy-1.24.4-cp38-cp38-macosx_10_12_arm64.whl is not a supported wheel on this platform.

What does the macOS toolchain on arm64 do when $MACOSX_DEPLOYMENT_TARGET is set to a version smaller than 11.0? Does it silently upgrade it to 11.0 or does it ignore the environment variable completely?

It silently upgrades it. Tried with setting the env var to 10.12 on PyWavelets, which produces a wheel with 10_12 in the name. Then unzipped it and check an extension module for minos:

% otool -l _cwt.cpython-310-darwin.so | rg -A4 LC_BUILD_VERSION
      cmd LC_BUILD_VERSION
  cmdsize 32
 platform 1
    minos 11.0
      sdk 14.0

That also shows the problem I had in mind when I said "fragile". The SO answer I linked higher up no longer works, the otool output changed so it has to be parsed differently.

I think that we should make the easy change here, ensuring arm64 wheel names target 11.0 or later. But not parse all binaries, that's the job of the user to not make mistakes here. It's not difficult to set the env var to the value you need I'd say.

[QuLogic]

I am not sure that setting $MACOSX_DEPLOYMENT_TARGET to something invalid should get special treatment by meson-python. I would have expected the macOS toolchain to error out if the env var is set to something that does not make sense.

Note that what's described here is not necessarily invalid; the MACOSX_DEPLOYMENT_TARGET also affects the API that you are able to use. So one might set it older in order to limit macOS API, but as noted, the Xcode toolchain will still record a newer minimum runtime ABI version in the binary.

Yes, I double checked and pip just refuses, which is arguably a pip bug, but that ship has sailed a long time ago probably (didn't search their issue tracker to verify):

From what we could understand of the code when we looked into it yesterday, pip doesn't parse the platform tag from the wheel and check it. Rather, it iterates through valid platform tags in order to match the filename, and when generating "valid platform tags", it starts at 11.0 on M1.

[rgommers]

Ah yes, right - that "iterate through valid tags" has bitten us before, it's probably fast but not all that healthy. I'd still say it's a pip bug though, since how they do the checking under the hood has to be an implementation detail and cannot prescribe what's a valid wheel name. Anway, we're going to have to work around that one.

[QuLogic]

Any chance we could get a release with this fixed out soon? I don't see anything on the 0.16.0 milestone that's open.