Bug: MACOSX_DEPLOYMENT_TARGET is not respected

[ilya-lavrenov]

Command to build project:

MACOSX_DEPLOYMENT_TARGET=11.0 python3 -m pip wheel -v --no-deps --wheel-dir wheels .

Logs and final package:

  Running command Preparing metadata (pyproject.toml)
  loading initial cache file /Users/sandye51/Documents/Programming/git_repo/openvino_tokenizers/.py-build-cmake_cache/cp312-cp312-macosx_14_0_arm64/py-build-cmake-preload.cmake
....
  Created wheel for openvino-tokenizers: filename=openvino_tokenizers-2024.3.0.0-py3-none-macosx_14_0_arm64.whl size=44940 sha256=4ebce1f2cf2d12e880f4274c9b029004a31e71e6d9f66aec94a3addd09150032
  Stored in directory: /private/var/folders/d_/z_lpvwrx1h37bv_k0m5mf1yr0000gn/T/pip-ephem-wheel-cache-7ct8ndqt/wheels/99/f4/22/a91c0bf10aabb4e3ce24bcf9d46d57e94cf8713d37c740a312

It can be overridden via extra option --config-settings=override=cross.arch="macosx_11_0_arm64"

But some other projects like scikit-build-core respect this option

[tttapa]

This appears to be another distlib limitation. py-build-cmake uses distlib.util.get_platform().

Tools like cibuildwheel set the _PYTHON_HOST_PLATFORM environment variable, e.g. _PYTHON_HOST_PLATFORM='macosx-11.0-arm64', which does get respected.

I'll have a look at implementing some MACOSX_DEPLOYMENT_TARGET-awareness in py-build-cmake.

[ilya-lavrenov]

I'll have a look at implementing some MACOSX_DEPLOYMENT_TARGET-awareness in py-build-cmake.

Would be great to support it.

Overall, most of tickets I create - respect actual build env in wheel meta info / file name 😺

BTW, one of the benefits I see in py-build-cmake is that we don't need to specify _PYTHON_HOST_PLATFORM, which in case of cross-compilation leads to installation of build requirements for host system. And hence, cmake installed as host requirement, cannot be run on build platform.

[tttapa]

The main problem here is that py-build-cmake can only try to make a guess: Having MACOSX_DEPLOYMENT_TARGET set in the environment provides no guarantee whatsoever that the package will actually run on such a target. For example, some of the dependencies may be compiled with later versions of macOS, or MACOSX_DEPLOYMENT_TARGET may not have been set during a previous CMake run, and the cached object files with the wrong version are re-used.

Given this situation, I feel like the only safe thing that py-build-cmake can do is simply use the current interpreter's value of distlib.util.get_platform() and ignore any environment variables that may or may not have been set while the code and/or dependencies were built. Either that, or the user should tell py-build-cmake explicitly what the correct platform tag should be, and then all the onus is on the user (or on a tool cibuildwheel).

What are your thoughts on this?

[ilya-lavrenov]

What are your thoughts on this?

My thought is that as an user, I don't want to think that internally some packages like distlib provide wrong values and don't respect my environment. When I compile python extension with a given MACOSX_DEPLOYMENT_TARGET value, I want all build stack to respect it. But currently, compiler respects this value, while packaging tool does not do it. And, hence, I need to perform post-processing of resulting wheel package or set the same information multiple time (I mean --config-settings=override=cross.arch="macosx_11_0_arm64").

IMO, MACOSX_DEPLOYMENT_TARGET is a well-known standard on macOS and should be respected by all the tool.

For example, some of the dependencies may be compiled with later versions of macOS

Why should we consider weird cases (when non-experienced developers mix wrong binaries) as default case? Default case - out of box convenience. Even now, users can build on macOS 12 with some dependences from macOS 14 and platform tag will be detected as macosx_12_0_arm64.