search

Quansight-Labs / free-threaded-compatibility

A central repository to keep track of the status of work on and support for free-threaded CPython (see PEP 703), with a focus on the scientific and ML/AI ecosystem
https://py-free-threading.github.io/
MIT License
155 stars 22 forks source link

Status of packaging tools #7

Open rgommers opened 7 months ago

rgommers commented 7 months ago
henryiii commented 5 months ago
rgommers commented 5 months ago

Thanks @henryiii! Table updated.

henryiii commented 5 months ago

Looks like both meson and CMake don't support Windows + free-threading yet, they don't know to look for the library with the "t" for linking. I think I can fix this in scikit-build-core if I can tell it the correct library from sysconfig info.

rgommers commented 5 months ago

Yeah, working on the same thing for Meson. I think adding the t in python313t.lib based on:

bool(sysconfig.get_config_var('Py_GIL_DISABLED'))

should do the job. Plus the debug flavor of it (there's no limited API variant).

henryiii commented 5 months ago

Windows free-threaded fixed in scikit-build-core 0.9.5.

henryiii commented 5 months ago

FYI, sysconfig's LIBRARY is correct as far as the "t" goes but is .dll instead of .lib.

henryiii commented 5 months ago

Free-threading including macOS with 3.13.0b2 fully supported in cibuildwheel 2.19.0.

rgommers commented 4 months ago

The final release of Pip 24.1 is out 🎉. That was the biggest missing piece here, we're in a pretty happy place now.

rgommers commented 4 months ago

The Windows support in Meson by fixed in https://github.com/mesonbuild/meson/pull/13338. @henryiii there is a second issue on Windows beyond libpython, namely that pyconfig.h is missing the Py_GIL_DISABLED define (discussed in https://discuss.python.org/t/windows-installer-freethreading-and-building-extension-modules/54391/2). I only lightly browsed your CMake/scikit-build-core PRs, so maybe I missed it - but it wasn't clear to me that that fix was included.

EDIT: xref https://gitlab.kitware.com/cmake/cmake/-/issues/26016 for CMake Windows issue.

minrk commented 3 months ago

Is it appropriate to add tracking of pyO3 and cffi here in build backends?

I was testing Jupyter today, and of the extension dependencies for JupyterLab, these appear to both be limiting factors. Most extensions are working fine, though:

so I can run a nogil kernel, but not the UI yet.

rgommers commented 3 months ago

Hey @minrk, thanks for the input! I'd say that these two aren't packaging tools so this isn't quite the right issue - and they're both important enough to deserve their own tracking issue.

PyO3 working will take a while, work has just started and it's quite complex. @ngoldbaum perhaps you could open a separate tracking issue for this?

cffi hasn't moved since December I think (aside from the PR you just opened), and it's the first time it has come up here. Maybe you want to open a new issue for it on this repo?

ngoldbaum commented 3 months ago

I opened https://github.com/Quansight-Labs/free-threaded-compatibility/issues/53 for PyO3, thanks for the reminder.

minrk commented 3 months ago

Thanks, I somehow overlooked that pybind11 is not in this list, which is what I was thinking of as a peer for these. Opened #54 for cffi.

henryiii commented 2 weeks ago

but it wasn't clear to me that that fix was included.

FYI, this was fixed in CMake 3.30.3 a month or two ago.

rgommers commented 2 weeks ago

Great, thanks @henryiii!

rgommers commented 2 weeks ago

Windows wheels are also starting to materialize and that all looks good, so I ticked off delvewheel. I'm sure there are some corner case bugs lurking somewhere, but overall we're in good shape here.