python / cpython

The Python programming language
https://www.python.org
Other
63.19k stars 30.26k forks source link

Running Cython fails with 3.14.0a1 - a list attribute gets changed to `None` #125868

Open da-woods opened 2 days ago

da-woods commented 2 days ago

Bug report

Bug description:

To be clear - this issue is just about running the Cython itself (i.e. pure Python code). It is not about compiled extension modules.

To reproduce

You get an output that ends with

File 'ExprNodes.py', line 8643, in analyse_types: TupleNode(Parsing.py:750:34,
    is_sequence_constructor = 1,
    result_is_used = True,
    use_managed_ref = True)

Compiler crash traceback from this point on:
  File "/home/dave/Documents/programming/cython2/Cython/Compiler/ExprNodes.py", line 8643, in analyse_types
    if len(self.args) == 0:
       ~~~^^^^^^^^^^^
TypeError: object of type 'NoneType' has no len()

To try to debug it some more I add a constructor to TupleNode (in Cython/Compiler/ExprNodes.py at line 8627) with a breakpoint:

def __init__(self, pos, **kw):
        if "args" in kw and kw['args'] is None:
            breakpoint()
        super().__init__(pos, **kw)
> <fullpath>/Cython/Compiler/ExprNodes.py(8629)__init__()
-> breakpoint()
(Pdb) print(kw)
{'args': None}
(Pdb) up
> <fullpath>/Cython/Compiler/ExprNodes.py(6123)analyse_types()
-> self.arg_tuple = TupleNode(self.pos, args = self.args)
(Pdb) print(self.args)
[<Cython.Compiler.ExprNodes.NameNode object at 0x7fb636188c80>]
(Pdb) 

So the the constructor call to TupleNode args is None. But in the function it's being called from it's a list containing a NameNode (which is what I think it should be). That's as far as I've got with debugging.

This has apparently been bisected to https://github.com/python/cpython/pull/122620 (but not by me).

CPython versions tested on:

3.14

Operating systems tested on:

Linux

Linked PRs

markshannon commented 2 days ago

My guess is that https://github.com/python/cpython/pull/122620 is to blame. Specifically removing this deopt https://github.com/python/cpython/pull/122620/files#diff-729a985b0cb8b431cb291f1edb561bbbfea22e3f8c262451cd83328a0936a342L2221

If a value is NULL, then the usual invariants on keys might not apply to that key.

Suppose we had a dict {'a': 'b'} with a table looking like this: key value
"a" "b"
"a" NULL

I'm not sure how such a table would come into being, but it might be possible through a convoluted sequence of additions and deletions

If a STORE_ATTR_WITH_HINT had index=1 and key="a" then it would change the table to key value
"a" "b"
"a" NEW_VALUE

which definitely isn't legal.

leftys commented 2 days ago

I tried to compile numpy from @markshannon fix branch and the Cython compilation suddenly worked. Thank you!