microsoft / LightGBM

A fast, distributed, high performance gradient boosting (GBT, GBDT, GBRT, GBM or MART) framework based on decision tree algorithms, used for ranking, classification and many other machine learning tasks.
https://lightgbm.readthedocs.io/en/latest/
MIT License
16.56k stars 3.82k forks source link

[ci] [python-package] run macOS x86_64 tests on macOS 12 (Monterey), stop manually setting macOS wheel tags #6487

Closed jameslamb closed 3 months ago

jameslamb commented 3 months ago

Description

Follow-up to #5328

Proposes some changes to macOS packaging:

Notes for Reviewers

Why drop Big Sur testing?

Azure DevOps is dropping support for macOS Big Sur (11.x). See these warnings at the top of all the recent Azure DevOps builds here:

Following macOS 11 End of Support by Apple, macOS 11 images will be removed on 28th June.

(example build link)

According to https://endoflife.date/macos, Big Sur support ended in September 2023

image

List of supported Microsoft-hosted Azure DevOps images: https://learn.microsoft.com/en-us/azure/devops/pipelines/agents/hosted?view=azure-devops&tabs=yaml#use-a-microsoft-hosted-agent

Why let scikit-build-core choose the platform target?

This would result in supporting a wider range of macOS versions with lightgbm's wheels than the last release (v4.4.0) did. For example, in 4.4.40 lightgbm has this:

# arm64: macOS >= 14.0
lightgbm-4.4.0-py3-none-macosx_14_0_arm64.whl 

# x86_64: macOS >= 10.15
lightgbm-4.4.0-py3-none-macosx_10_15_x86_64.macosx_11_6_x86_64.macosx_12_5_x86_64.whl 

ref: https://pypi.org/project/lightgbm/4.4.0/#files

As of this PR, lightgbm would support:

# arm64: macOS >= 12.0
lightgbm-4.4.0-py3-none-macosx_12_0_arm64.whl

# x86_64: macOS >= 10.15
lightgbm-4.4.0-py3-none-macosx_10_15_x86_64.whl

CI logs:

How does automatic determination of the tag work?

scikit-build-core determines the range of macOS versions by inspecting the MACOSX_DEPLOYMENT_TARGET environment variable where available, and otherwise falling back to platform.mac_ver().

https://github.com/scikit-build/scikit-build-core/blob/acb7d0346e4a05bcb47a4ea3939c705ab71e3145/src/scikit_build_core/builder/macos.py#L36

Using this mechanism instead of manual renaming allows building wheels that target macOS versions older than whatever version the CI build is running on, to continue to support users on older systems. This is how xgboost, for example, is able to continue producing wheels that support macOS 10.X (Catalina).

https://github.com/dmlc/xgboost/blob/6c83c8c2efdb522472c1d23349ab64320d455443/tests/ci_build/build_python_wheels.sh#L38

https://pypi.org/project/xgboost/2.0.3/#files

It should also be less error-prone than manually setting tags by renaming files.