search

mesonbuild / meson-python

Meson PEP 517 Python build backend
https://mesonbuild.com/meson-python/
MIT License
120 stars 59 forks source link

Need to augment wheel tag computation code to account for nogil builds #499

Closed dnicolodi closed 1 month ago

dnicolodi commented 9 months ago

Starting with Python 3.13 there will be a nogil variant of the interpreter that will need to be get an interpreter tag in the wheel tag string. See https://discuss.python.org/t/python-abis-and-pep-703/34018

dnicolodi commented 6 months ago

See also https://github.com/pypa/packaging/issues/727 https://github.com/pypa/packaging/pull/728 https://github.com/pypa/packaging/pull/747

rgommers commented 2 months ago

gh-621 addresses part of this. As of today, it's not yet very easy to test with a free-threaded CPython build and make the test suite pass. The hiccups are:

In order to make pip install wheels with the t tag, the _fail_if_link_is_unsupported_wheel function in pip/_internal/resolution/resolvelib/factory.py needs to be patched - simply replacing raise UnsupportedWheel(msg) with return is enough.

Then the venv fixture in our test suite needs patching, because it installs a fresh copy of pip in the venv's it creates. This hacky patch does that (replace path to site-packages as needed):

diff --git a/tests/conftest.py b/tests/conftest.py
index 80303a14f..68df9e2ef 100644
--- a/tests/conftest.py
+++ b/tests/conftest.py
@@ -101,7 +101,14 @@ class VEnv(EnvBuilder):
 @pytest.fixture()
 def venv(tmp_path_factory):
     path = pathlib.Path(tmp_path_factory.mktemp('mesonpy-test-venv'))
-    return VEnv(path)
+    fixture = VEnv(path)
+
+    # Apply fix to `pip` installed into venv to avoid unknown tag issue
+    sitepkg_dir = '/home/rgommers/mambaforge/envs/nogil/lib/python3.13/site-packages'
+    sitepkg_venv = str(path) + '/lib/python3.13/site-packages'
+    _file = '/pip/_internal/resolution/resolvelib/factory.py'
+    subprocess.run(['cp', sitepkg_dir + _file, sitepkg_venv + _file])
+    return fixture

 def generate_package_fixture(package):
dnicolodi commented 2 months ago

gh-621 addresses part of this

@rgommers With #621 merged, what is left to do to support nogil Python?

rgommers commented 2 months ago

Everything works, I only left this open to remind myself to add a CI job once the setup-python action starts offering a free-threaded CPython.

Happy to close this though and keep track of that task elsewhere - it's not urgent, and we're unlikely to regress on full support.

dnicolodi commented 2 months ago

Do you have a sense for when we may see nogil builds available via actions/setup-python? The alpha releases of Python 3.13 are available, but no nogil builds.

rgommers commented 2 months ago

Tracked in https://github.com/actions/setup-python/issues/771. No timeline, but I think the desire is to do that for the 3.13.0b1. So hopefully in May.

dnicolodi commented 2 months ago

Oh, you mentioned the actions/setup-python bug in #621. I noticed it when I read the PR but then I completely forgot about it, sorry. Our test matrix is becoming very large. I'm wondering if there is something that would be reasonable to drop.

rgommers commented 2 months ago

We should probably be able to drop some Meson versions? If we test oldest/newest releases plus master branch, that gets us pretty much what we need. Unless we have a specific reason to test every minor version. We can also combine Meson and Python versions, rather than having separate jobs for "check each Python version with latest meson" and then "check each meson version".

henryiii commented 1 month ago

You can use python3.13t in manylinux2014+ starting today.

rgommers commented 1 month ago

Awesome. I wasn't expecting that just yet, but it looks like it was enabled by including pip 24.1b1 in the image (see https://github.com/pypa/manylinux/pull/1564).

dnicolodi commented 1 month ago

I'm giving it a try just now, see #627

dnicolodi commented 1 month ago

Tests for free-threading Python 3.13 are up and running. Closing this issue.