search

MagicStack / immutables

A high-performance immutable mapping type for Python.
Other
1.13k stars 57 forks source link

Segfault with Python 3.13.0b1 #116

Closed mgorny closed 1 month ago

mgorny commented 6 months ago 
$ export CFLAGS='-Og -g -Wall'
$ python3.13 setup.py build_ext -i
running build_ext
building 'immutables._map' extension
creating build
creating build/temp.linux-x86_64-cpython-313
creating build/temp.linux-x86_64-cpython-313/immutables
x86_64-pc-linux-gnu-gcc -fno-strict-overflow -Wsign-compare -Og -g -Wall -fPIC -DNDEBUG=1 -I/usr/include/python3.13 -c immutables/_map.c -o build/temp.linux-x86_64-cpython-313/immutables/_map.o -O2 -std=c99 -fsigned-char -Wall -Wsign-compare -Wconversion
immutables/_map.c: In function ‘map_node_bitmap_dump’:
immutables/_map.c:1287:12: warning: implicit declaration of function ‘_PyLong_Format’; did you mean ‘_PyLong_Copy’? [-Wimplicit-function-declaration]
 1287 |     tmp2 = _PyLong_Format(tmp1, 2);
      |            ^~~~~~~~~~~~~~
      |            _PyLong_Copy
immutables/_map.c:1287:10: warning: assignment to ‘PyObject *’ {aka ‘struct _object *’} from ‘int’ makes pointer from integer without a cast [-Wint-conversion]
 1287 |     tmp2 = _PyLong_Format(tmp1, 2);
      |          ^
creating build/lib.linux-x86_64-cpython-313
creating build/lib.linux-x86_64-cpython-313/immutables
x86_64-pc-linux-gnu-gcc -shared -Og -g -Wall build/temp.linux-x86_64-cpython-313/immutables/_map.o -L/usr/lib64 -o build/lib.linux-x86_64-cpython-313/immutables/_map.cpython-313-x86_64-linux-gnu.so
copying build/lib.linux-x86_64-cpython-313/immutables/_map.cpython-313-x86_64-linux-gnu.so -> immutables
$ python -m pytest -s
========================================================= test session starts =========================================================
platform linux -- Python 3.13.0b1, pytest-8.2.0, pluggy-1.5.0
rootdir: /tmp/immutables
configfile: pyproject.toml
testpaths: tests
collected 159 items                                                                                                                   

tests/test_issue24.py .....Fatal Python error: Segmentation fault

Current thread 0x00007f08dab04740 (most recent call first):
  File "/tmp/immutables/tests/test_issue24.py", line 121 in hamt_dump_check_first_return_second
  File "/tmp/immutables/tests/test_issue24.py", line 141 in test_array_node_delete_in_place_count
  File "/usr/lib/python3.13/unittest/case.py", line 606 in _callTestMethod
  File "/usr/lib/python3.13/unittest/case.py", line 651 in run
  File "/usr/lib/python3.13/unittest/case.py", line 707 in __call__
  File "/tmp/venv/lib/python3.13/site-packages/_pytest/unittest.py", line 343 in runtest
  File "/tmp/venv/lib/python3.13/site-packages/_pytest/runner.py", line 173 in pytest_runtest_call
  File "/tmp/venv/lib/python3.13/site-packages/pluggy/_callers.py", line 103 in _multicall
  File "/tmp/venv/lib/python3.13/site-packages/pluggy/_manager.py", line 120 in _hookexec
  File "/tmp/venv/lib/python3.13/site-packages/pluggy/_hooks.py", line 513 in __call__
  File "/tmp/venv/lib/python3.13/site-packages/_pytest/runner.py", line 241 in <lambda>
  File "/tmp/venv/lib/python3.13/site-packages/_pytest/runner.py", line 341 in from_call
  File "/tmp/venv/lib/python3.13/site-packages/_pytest/runner.py", line 240 in call_and_report
  File "/tmp/venv/lib/python3.13/site-packages/_pytest/runner.py", line 135 in runtestprotocol
  File "/tmp/venv/lib/python3.13/site-packages/_pytest/runner.py", line 116 in pytest_runtest_protocol
  File "/tmp/venv/lib/python3.13/site-packages/pluggy/_callers.py", line 103 in _multicall
  File "/tmp/venv/lib/python3.13/site-packages/pluggy/_manager.py", line 120 in _hookexec
  File "/tmp/venv/lib/python3.13/site-packages/pluggy/_hooks.py", line 513 in __call__
  File "/tmp/venv/lib/python3.13/site-packages/_pytest/main.py", line 364 in pytest_runtestloop
  File "/tmp/venv/lib/python3.13/site-packages/pluggy/_callers.py", line 103 in _multicall
  File "/tmp/venv/lib/python3.13/site-packages/pluggy/_manager.py", line 120 in _hookexec
  File "/tmp/venv/lib/python3.13/site-packages/pluggy/_hooks.py", line 513 in __call__
  File "/tmp/venv/lib/python3.13/site-packages/_pytest/main.py", line 339 in _main
  File "/tmp/venv/lib/python3.13/site-packages/_pytest/main.py", line 285 in wrap_session
  File "/tmp/venv/lib/python3.13/site-packages/_pytest/main.py", line 332 in pytest_cmdline_main
  File "/tmp/venv/lib/python3.13/site-packages/pluggy/_callers.py", line 103 in _multicall
  File "/tmp/venv/lib/python3.13/site-packages/pluggy/_manager.py", line 120 in _hookexec
  File "/tmp/venv/lib/python3.13/site-packages/pluggy/_hooks.py", line 513 in __call__
  File "/tmp/venv/lib/python3.13/site-packages/_pytest/config/__init__.py", line 178 in main
  File "/tmp/venv/lib/python3.13/site-packages/_pytest/config/__init__.py", line 206 in console_main
  File "/tmp/venv/lib/python3.13/site-packages/pytest/__main__.py", line 7 in <module>
  File "<frozen runpy>", line 88 in _run_code
  File "<frozen runpy>", line 198 in _run_module_as_main
Segmentation fault (core dumped)
$ coredumpctl gdb -1
[…]
#0  0x00007f08da29d2bc in ?? () from /usr/lib64/libc.so.6
#1  0x00007f08da24b926 in raise () from /usr/lib64/libc.so.6
#2  <signal handler called>
#3  0x00007f08da5f3a11 in PyObject_Str () from /usr/lib64/libpython3.13.so.1.0
#4  0x00007f08da5a6faf in PyUnicode_FromFormatV () from /usr/lib64/libpython3.13.so.1.0
#5  0x00007f08d8df7ef2 in _map_dump_format (writer=writer@entry=0x7ffe833cc340, 
    format=format@entry=0x7f08d8dfd03e "bitmap=%S id=%p):\n") at immutables/_map.c:537
#6  0x00007f08d8df8e44 in map_node_bitmap_dump (level=1, writer=0x7ffe833cc340, node=0x7f08d758cdb0) at immutables/_map.c:1292
#7  map_node_dump (node=0x7f08d758cdb0, writer=writer@entry=0x7ffe833cc340, level=level@entry=1) at immutables/_map.c:2261
#8  0x00007f08d8df8d71 in map_node_array_dump (level=0, writer=0x7ffe833cc340, node=0x7f08d7d4f480) at immutables/_map.c:2127
#9  map_node_dump (node=0x7f08d7d4f480, writer=writer@entry=0x7ffe833cc340, level=level@entry=0) at immutables/_map.c:2265
#10 0x00007f08d8df8f9f in map_dump (self=0x7f08d7563140) at immutables/_map.c:2620
#11 map_py_dump (self=0x7f08d7563140, args=<optimized out>) at immutables/_map.c:3191
#12 0x00007f08da5ddf97 in ?? () from /usr/lib64/libpython3.13.so.1.0
#13 0x00007f08da5a51b5 in PyObject_Vectorcall () from /usr/lib64/libpython3.13.so.1.0
#14 0x00007f08da5bb637 in _PyEval_EvalFrameDefault () from /usr/lib64/libpython3.13.so.1.0
#15 0x00007f08da601f66 in ?? () from /usr/lib64/libpython3.13.so.1.0
#16 0x00007f08da5f0427 in ?? () from /usr/lib64/libpython3.13.so.1.0
#17 0x00007f08da5bd27e in _PyEval_EvalFrameDefault () from /usr/lib64/libpython3.13.so.1.0
#18 0x00007f08da5a5dea in ?? () from /usr/lib64/libpython3.13.so.1.0
#19 0x00007f08da5e7301 in ?? () from /usr/lib64/libpython3.13.so.1.0
#20 0x00007f08da6ac8f3 in ?? () from /usr/lib64/libpython3.13.so.1.0
#21 0x00007f08da5a0db4 in _PyObject_MakeTpCall () from /usr/lib64/libpython3.13.so.1.0
#22 0x00007f08da5c3963 in _PyEval_EvalFrameDefault () from /usr/lib64/libpython3.13.so.1.0
#23 0x00007f08da5a5dea in ?? () from /usr/lib64/libpython3.13.so.1.0
#24 0x00007f08da5e7301 in ?? () from /usr/lib64/libpython3.13.so.1.0
#25 0x00007f08da6ac8f3 in ?? () from /usr/lib64/libpython3.13.so.1.0
#26 0x00007f08da5f0289 in ?? () from /usr/lib64/libpython3.13.so.1.0
#27 0x00007f08da5bd27e in _PyEval_EvalFrameDefault () from /usr/lib64/libpython3.13.so.1.0
#28 0x00007f08da5a5dea in ?? () from /usr/lib64/libpython3.13.so.1.0
#29 0x00007f08da5e7301 in ?? () from /usr/lib64/libpython3.13.so.1.0
#30 0x00007f08da6ac8f3 in ?? () from /usr/lib64/libpython3.13.so.1.0
#31 0x00007f08da5a0db4 in _PyObject_MakeTpCall () from /usr/lib64/libpython3.13.so.1.0
#32 0x00007f08da5c3963 in _PyEval_EvalFrameDefault () from /usr/lib64/libpython3.13.so.1.0
#33 0x00007f08da5a5dea in ?? () from /usr/lib64/libpython3.13.so.1.0
#34 0x00007f08da5e7301 in ?? () from /usr/lib64/libpython3.13.so.1.0
#35 0x00007f08da6ac8f3 in ?? () from /usr/lib64/libpython3.13.so.1.0
#36 0x00007f08da5a0db4 in _PyObject_MakeTpCall () from /usr/lib64/libpython3.13.so.1.0
#37 0x00007f08da5c3963 in _PyEval_EvalFrameDefault () from /usr/lib64/libpython3.13.so.1.0
#38 0x00007f08da5a5dea in ?? () from /usr/lib64/libpython3.13.so.1.0
#39 0x00007f08da5e7301 in ?? () from /usr/lib64/libpython3.13.so.1.0
#40 0x00007f08da6ac8f3 in ?? () from /usr/lib64/libpython3.13.so.1.0
#41 0x00007f08da5a0db4 in _PyObject_MakeTpCall () from /usr/lib64/libpython3.13.so.1.0
#42 0x00007f08da5c3963 in _PyEval_EvalFrameDefault () from /usr/lib64/libpython3.13.so.1.0
#43 0x00007f08da6590ad in PyEval_EvalCode () from /usr/lib64/libpython3.13.so.1.0
#44 0x00007f08da67469b in ?? () from /usr/lib64/libpython3.13.so.1.0
#45 0x00007f08da5cfcc6 in ?? () from /usr/lib64/libpython3.13.so.1.0
#46 0x00007f08da5a51b5 in PyObject_Vectorcall () from /usr/lib64/libpython3.13.so.1.0
#47 0x00007f08da5bb637 in _PyEval_EvalFrameDefault () from /usr/lib64/libpython3.13.so.1.0
#48 0x00007f08da68e1ec in ?? () from /usr/lib64/libpython3.13.so.1.0
#49 0x00007f08da68d9a3 in Py_RunMain () from /usr/lib64/libpython3.13.so.1.0
#50 0x00007f08da6411bb in Py_BytesMain () from /usr/lib64/libpython3.13.so.1.0
#51 0x00007f08da235350 in ?? () from /usr/lib64/libc.so.6
#52 0x00007f08da235409 in __libc_start_main () from /usr/lib64/libc.so.6
#53 0x0000562f9225b085 in _start ()
musicinmybrain commented 5 months ago

Note that this

immutables/_map.c: In function ‘map_node_bitmap_dump’:
immutables/_map.c:1287:12: warning: implicit declaration of function ‘_PyLong_Format’; did you mean ‘_PyLong_Copy’? [-Wimplicit-function-declaration]
 1287 |     tmp2 = _PyLong_Format(tmp1, 2);
      |            ^~~~~~~~~~~~~~
      |            _PyLong_Copy
immutables/_map.c:1287:10: warning: assignment to ‘PyObject *’ {aka ‘struct _object *’} from ‘int’ makes pointer from integer without a cast [-Wint-conversion]
 1287 |     tmp2 = _PyLong_Format(tmp1, 2);
      |          ^

reflects the fact that _PyLong_Format belongs to the internal, private API, and is now defined in internal/pycore_long.h and only when Py_BUILD_CORE is defined. Extension modules can do this, but these APIs are supposed to be only for building Python itself.

Note also that in Fedora Rawhide, implicit function declarations like this are errors rather than warnings.

What’s more, replacing _PyLong_Format with PyNumber_ToBase seems to be enough for the tests to pass on Python 3.13.0b1.

sodul commented 1 month ago

I just tested with Python 3.13.0rc2 and immutables==0.20. I can confirm that the issue is still present.

mgorny commented 1 month ago

Thank you!