search

geopandas / pyogrio

Vectorized vector I/O using OGR
https://pyogrio.readthedocs.io
MIT License
257 stars 21 forks source link

BLD/RLS: Bump MacOS wheels to MacOS 11 (+ bump cibuildwheel from 2.17.0 to 2.18.1) #417

Closed dependabot[bot] closed 3 weeks ago

dependabot[bot] commented 1 month ago

Bumps pypa/cibuildwheel from 2.17.0 to 2.18.1.

Release notes

Sourced from pypa/cibuildwheel's releases.

v2.18.1

  • 🌟 Add free-threaded Linux and Windows builds for 3.13. New identifiers cp313t-*, new option CIBW_FREE_THREADED_SUPPORT/tool.cibuildwheel.free-threaded-support required to opt-in. See the docs for more information. (#1831)
  • ✨ The container-engine is now a build (non-global) option. (#1792)
  • 🛠 The build backend for cibuildwheel is now hatchling. (#1297)
  • 🛠 Significant improvements and modernization to our noxfile. (#1823)
  • 🛠 Use pylint's new GitHub Actions reporter instead of a custom matcher. (#1823)
  • 🛠 Unpin virtualenv updates for Python 3.7+ (#1830)
  • 🐛 Fix running linux tests from Windows or macOS ARM. (#1788)
  • 📚 Fix our documentation build. (#1821)

v2.18.0

  • ✨ Adds CPython 3.13 support, under the prerelease flag CIBW_PRERELEASE_PYTHONS. This version of cibuildwheel uses 3.13.0b1. Free-threading mode is not available yet (#1657), waiting on official binaries (planned for beta 2) and pip support. While CPython is in beta, the ABI can change, so your wheels might not be compatible with the final release. For this reason, we don't recommend distributing wheels until RC1, at which point 3.13 will be available in cibuildwheel without the flag. (#1815)
  • ✨ Musllinux now defaults to musllinux_1_2. You can set the older musllinux_1_1 via config if needed. (#1817)
  • 🛠 No longer pre-seed setuptools/wheel in virtual environments (#1819)
  • 🛠 Respect the constraints file when building with pip, matching build (#1818)
  • 🛠 Use uv to compile our pinned dependencies, 10x faster and doesn't require special setup (#1778)
  • 🐛 Fix an issue with the schema (#1788)
  • 📚 Document the new delocate error checking macOS versions (#1766)
  • 📚 Document Rust builds (#1816)
  • 📚 Speed up our readthedocs builds with uv, 26 seconds -> 6 seconds to install dependencies (#1816)
Changelog

Sourced from pypa/cibuildwheel's changelog.

v2.18.1

  • 🌟 Add free-threaded Linux and Windows builds for 3.13. New identifiers cp313t-*, new option CIBW_FREE_THREADED_SUPPORT/tool.cibuildwheel.free-threaded-support required to opt-in. See the docs for more information. (#1831)
  • ✨ The container-engine is now a build (non-global) option. (#1792)
  • 🛠 The build backend for cibuildwheel is now hatchling. (#1297)
  • 🛠 Significant improvements and modernization to our noxfile. (#1823)
  • 🛠 Use pylint's new GitHub Actions reporter instead of a custom matcher. (#1823)
  • 🛠 Unpin virtualenv updates for Python 3.7+ (#1830)
  • 🐛 Fix running linux tests from Windows or macOS ARM. (#1788)
  • 📚 Fix our documentation build. (#1821)

v2.18.0

12 May 2024

  • ✨ Adds CPython 3.13 support, under the prerelease flag CIBW_PRERELEASE_PYTHONS. This version of cibuildwheel uses 3.13.0b1. Free-threading mode is not available yet (#1657), waiting on official binaries (planned for beta 2) and pip support.

    While CPython is in beta, the ABI can change, so your wheels might not be compatible with the final release. For this reason, we don't recommend distributing wheels until RC1, at which point 3.13 will be available in cibuildwheel without the flag. (#1815)

  • ✨ Musllinux now defaults to musllinux_1_2. You can set the older musllinux_1_1 via config if needed. (#1817)

  • 🛠 No longer pre-seed setuptools/wheel in virtual environments (#1819)

  • 🛠 Respect the constraints file when building with pip, matching build (#1818)

  • 🛠 Use uv to compile our pinned dependencies, 10x faster and doesn't require special setup (#1778)

  • 🐛 Fix an issue with the schema (#1788)

  • 📚 Document the new delocate error checking macOS versions (#1766)

  • 📚 Document Rust builds (#1816)

  • 📚 Speed up our readthedocs builds with uv, 26 seconds -> 6 seconds to install dependencies (#1816)

Commits
  • ba8be0d Bump version: v2.18.1
  • 90dd476 doc: add documentation for the CIBW_FREE_THREADED_SUPPORT option
  • b613249 chore: pass a PythonConfiguration to install_cpython
  • 3992d57 feat: add option to opt-in free-threaded builds
  • 345467c feat: add support for free-threaded (no-gil) Python 3.13
  • 791e41c [Bot] Update dependencies (#1832)
  • 9d5f5e6 Make container-engine a build (non-global) option (#1792)
  • 1b354cf chore: move to hatchling (#1297)
  • d1a4c9c chore: un-pin virtualenv update (#1830)
  • 78bca57 fix(tests): linux tests from macOS/Windows arm64 (#1829)
  • Additional commits viewable in compare view


Dependabot compatibility score

Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.

Dependabot commands and options
You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot merge` will merge this PR after your CI passes on it - `@dependabot squash and merge` will squash and merge this PR after your CI passes on it - `@dependabot cancel merge` will cancel a previously requested merge and block automerging - `@dependabot reopen` will reopen this PR if it is closed - `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually - `@dependabot show ignore conditions` will show all of the ignore conditions of the specified dependency - `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
jorisvandenbossche commented 3 weeks ago

With the latest versions, we get an error in delocate for MacOS complaining Library dependencies do not satisfy target MacOS version 10.9: . See https://github.com/pypa/cibuildwheel/pull/1766 for context.

The first commit setting the deployment target at 10.9 gave: /usr/local/share/vcpkg/buildtrees/gdal/src/v3.8.5-75371a500c.clean/port/cpl_vsi_mem.cpp:117:5: error: 'shared_mutex' is unavailable: introduced in macOS 10.12, so trying with 10.12 (it's not clear if GDAL requires a certain minimum version, so this is a bit trial and error)

jorisvandenbossche commented 3 weeks ago

Once I fixed the issue with having to specify the MACOSX_DEPLOYMENT_TARGET for the latest version of cibuildwheel and delocate, we run into the macOS x86_64 wheel segfaulting on import.

I had hoped that this update would have fixed that, but so those crashes are not related to original changes in this PR, since we see the segfaults in https://github.com/geopandas/pyogrio/pull/418 as well. The segfaults are probably related to some update on the macOS side (although it is strange this only happens now, given that the latest release in the macOS 11.x series already happened last year).

Specifically we see segfaults on macOS 11 with x86_64 (so not with the arm images). Several other projects are observing those segfaults as well, for example duckdb (https://github.com/duckdb/duckdb/issues/12199#issuecomment-2126992958, and they updated their python wheels to target 12.0 in https://github.com/duckdb/duckdb/pull/12236), and pyarrow (https://github.com/apache/arrow/issues/41696).

MacOS 11 (Big Sur) is no longer officially supported by Apple (although only 3.5 years old ..), and also the github actions macos-11 image will no longer be available starting next month (github-hosted runners).

So we will have to update the runner on which we build the x86_64 wheels anyway, but so also won't be able to test anymore if the wheels would be working on older macos versions. Many other projects (eg numpy) still release wheels tagged with 10.9 (like we also did up to now), although I don't think anyone tests that this actually works. But if we see that it actually does not even work on macOS 11.0, we should probably also bump the wheel tag to 12.0. This means that people trying to install pyogrio on an older macOS version will attempt to build from source because of no wheel available (they could still install an older version of pyogrio, for which the wheels might potentially still work). But the alternative is that we let the wheel install, potentially leading to a segfault (without possibility to test this in CI).

brendan-ward commented 3 weeks ago

I think given that MacOS 11 is no longer supported by Apple and the segfaults affect other projects within the ecosystem, it seems reasonable for us to bump the minimum supported version to 12, with the expectation that folks on 11 have to build from source (per DuckDB's approach) - rather than providing wheels that may segfault.

jorisvandenbossche commented 3 weeks ago

Yeah, with some trial an error, I tried to test whether it would still work if we build on newer macOS but target and test on older, but so that also gave segfaults. So given the limitations and the available runners for building our wheels on CI, I don't think we have another choice than building and targetting newer macOS.