python / cpython

The Python programming language
https://www.python.org
Other
63.64k stars 30.49k forks source link

3.14.0a2 rebuild needed for 0x03070000 limited api c extension #127191

Open MrBitBucket opened 2 days ago

MrBitBucket commented 2 days ago

Bug report

Bug description:

An existing C api extension compiled with Python 3.7-3.11 behaves differently when run in Python 3.14.0a2. Causing a reference count change where none is expected. The python test code looks like this

from sys import getrefcount
from _rl_accel import unicode2T1

utext = 'This is the end of the world'

class Font:
    def __init__(self,name):
        self.fontName = name
        self.substitutionFonts = []
        self.encName = 'utf8'

font = Font('Helvetica')

defns = 'utext font font.encName font.fontName font.substitutionFonts'.split()
rc0 = [getrefcount(eval(x,globals())) for x in defns]
print(rc0)
unicode2T1(utext,[font]+font.substitutionFonts)
rc1 = [getrefcount(eval(x,globals())) for x in defns]
print(rc1)
if rc1!=rc0:
    print('!!!!! reference counts changed !!!!!')

The C function unicode2T1 is not expected to change any of the refcounts checked, but if built with python 3.7-3.11 we see a decrease in the refcount for font.encName when running in python 3.14.0a2.

If built with python 3.14.0a2 the reference count decrease is not seen in the test. It seems that my code is broken in terms of the abi3 guarantee.

To ease checking I built a bash script to checkout and build the test environments; this can be found here.

I run this like

EARLYPYTHON=~/bin/python311 LATEPYTHON=~/bin/python314 tests/runner.sh

Sorry this example is so large, but at least reportlab is not required. We do have a python version of this code rl_accel.py

CPython versions tested on:

3.9, 3.10, 3.11, 3.12, 3.13, 3.14

Operating systems tested on:

Linux

MrBitBucket commented 1 day ago

I think this is expected now that the immortality of some/all globals is happening in Python >= 3.12. Probably I just need to build new wheels using a late python.

picnixz commented 1 day ago

I'm not sure we can do anything on our side since it affects 3rd party extensions and a security-branch only. However I don't know the exact policy concerning that kind of breaking change hence I'm asking @iritkatriel or @vstinner.

ZeroIntensity commented 1 day ago

As far as I know, the actual reference count of an object is an implementation detail, not part of the stable ABI.

vstinner commented 5 hours ago

Can you try to write a reproducer which doesn't depend on third party code?

vstinner commented 4 hours ago

Ah, this issue is also discussed at: https://discuss.python.org/t/3-14-0a2-rebuild-needed-for-0x03070000-limited-api-c-extension/72006