Open dimpase opened 10 months ago
I confirm this breaks some code out there.
In particular, we are unable to move from python 3.11 to anything higher, since we can no longer pickle.dump
our objects, even if sys.setrecursionlimit
is called beforehand for example.
Since 3.12,
sys.setrecursionlimit
does not control the recursion limit of the C stack (it's hardcoded inPy_C_RECURSION_LIMIT
, due to #91079), while the 3.13 docs sayThe fallout can be seen in #112215 (basically, any code that uses
lru_cache
in combination withsys.setrecursionlimit
is potentially, or actually, broken in 3.12, while still working in 3.11) or in #107263 (which was papered over with an increase ofPy_C_RECURSION_LIMIT
, not really fixed).This is an obvious change of behaviour, as code working in 3.11 can easily be broken in 3.12, as found in #112215
Such changes should carry a warning in the documentation.
Potentially, any recursive Python code combining C recursion (in a module, e.g.
lru_cache
, via@lru_cache
decorator) with increased (beyond the hardcodedPy_C_RECURSION_LIMIT
)sys.setrecursionlimit
is bound to be broken in 3.12.I believe the whole hardcoding of
Py_C_RECURSION_LIMIT
has to be fixed (perhaps with a separate API provided to change it), as it's a regression and an incompatible change of code behaviour, but meanwhile the docs have to reflect the reality.