search

python / cpython

The Python programming language
https://www.python.org/
Other
60.82k stars 29.36k forks source link

Freethreading interpreter will attempt to import unsupported stable ABI extension modules #120901

Open da-woods opened 1 week ago

da-woods commented 1 week ago

Bug report

Bug description:

Compile a stable ABI extension module with the filename "something.abi3.so" (or .pyd on Windows)

Run python -c "import something"

A freethreading build will attempt to import the extension module, even though it explicitly isn't compatible with the stable ABI. This (unsurprisingly) leads to a crash.

Tested on beta 3.13.0b2 (but a quick search didn't find relevant changes after that)

CPython versions tested on:

3.13

Operating systems tested on:

Linux

colesbury commented 1 week ago

The abi3.so or .pyd suffixes aren't enough to identify supported Python versions. For example, you can have an ABI 3.14+ library with abi3.so and Python 3.5 will load it and crash.

colesbury commented 5 days ago

I think we should probably refuse to load something.abi3.so in the free-threaded build, but I don't think we can do that for something.pyd on Windows (nor something.so on Unix).

@da-woods, can you provide more context on how you ran into this?

da-woods commented 5 days ago

but I don't think we can do that for something.pyd on Windows (nor something.so on Unix).

I didn't intend to propose this. I the sentence "(or .pyd on Windows)" was a bit ambiguous and I only meant ".abi3.pyd on Windows".

can you provide more context on how you ran into this?

I was testing an in-place no-gil build of Cython and hadn't properly cleared up all the .so files before trying to test something else with free-threading Python. This is obviously something you're more likely to run into when doing ad-hoc development of extension modules rather than using pre-compiled wheels.

colesbury commented 5 days ago

I only meant ".abi3.pyd on Windows"

I don't think Windows uses .abi3.pyd as a suffix, just .pyd for both ABI 3 shared libraries and unversioned shared libraries.

I was testing an in-place no-gil build of Cython...

That makes sense. I think that's related to https://github.com/python/cpython/issues/121103, where the free-threaded and default builds both use the same lib/python3.13 directory (and I think we should change that).