search

numpy / numpy

The fundamental package for scientific computing with Python.
https://numpy.org
Other
27.4k stars 9.76k forks source link

BUG: Segmentation fault on `TestMonsterType::test_dict_recursion` test Python3.12 main branch debug build #26939

Closed inakleinbottle closed 1 week ago

inakleinbottle commented 1 month ago

Describe the issue:

The nested dictionary dtype construction does not throw a RecursionError as expected in the test, and instead causes a segmentation fault. Since this is happening on a source build, I don't know if it is something that I have done to misconfigure the build, I just used spin build -- -Dbuildtype=debug and spin test.

Reproduce the code example:

spin build -- -Dbuildtype=debug
spin test

Error message:

spin test -- -v numpy/_core/tests/test_dtype.py::TestMonsterType::test_dict_recursion
Invoking `build` prior to running tests:
$ /home/sam/CLionProjects/numpy/venv/bin/python vendored-meson/meson/meson.py compile -C build
INFO: autodetecting backend as ninja
INFO: calculating backend command to run: /usr/bin/ninja -C /home/sam/CLionProjects/numpy/build
ninja: Entering directory `/home/sam/CLionProjects/numpy/build'
[1/1] Generating numpy/generate-version with a custom command
Saving version to numpy/version.py
$ /home/sam/CLionProjects/numpy/venv/bin/python vendored-meson/meson/meson.py install --only-changed -C build --destdir ../build-install
$ export PYTHONPATH="/home/sam/CLionProjects/numpy/build-install/usr/lib/python3.12/site-packages:/opt/intel/oneapi/advisor/2024.1/pythonapi"
$ export PYTHONPATH="/home/sam/CLionProjects/numpy/build-install/usr/lib/python3.12/site-packages"
$ /home/sam/CLionProjects/numpy/venv/bin/python -P -c 'import numpy'
$ cd /home/sam/CLionProjects/numpy/build-install/usr/lib/python3.12/site-packages
$ /home/sam/CLionProjects/numpy/venv/bin/python -P -m pytest --import-mode=importlib -m 'not slow' -v numpy/_core/tests/test_dtype.py::TestMonsterType::test_dict_recursion
============================================================================================================ test session starts =============================================================================================================
platform linux -- Python 3.12.4, pytest-8.2.2, pluggy-1.5.0 -- /home/sam/CLionProjects/numpy/venv/bin/python
cachedir: .pytest_cache
hypothesis profile 'np.test() profile' -> database=None, deadline=None, print_blob=True, derandomize=True, suppress_health_check=[HealthCheck.data_too_large, HealthCheck.filter_too_much, HealthCheck.too_slow, HealthCheck.large_base_example, HealthCheck.function_scoped_fixture, HealthCheck.differing_executors]
rootdir: /home/sam/CLionProjects/numpy
configfile: pytest.ini
plugins: hypothesis-6.107.0
collected 1 item

numpy/_core/tests/test_dtype.py::TestMonsterType::test_dict_recursion Fatal Python error: Segmentation fault

Current thread 0x00007ed684f25300 (most recent call first):
  File "/home/sam/CLionProjects/numpy/build-install/usr/lib/python3.12/site-packages/numpy/_core/tests/test_dtype.py", line 993 in test_dict_recursion
  File "/home/sam/CLionProjects/numpy/venv/lib/python3.12/site-packages/_pytest/python.py", line 162 in pytest_pyfunc_call
  File "/home/sam/CLionProjects/numpy/venv/lib/python3.12/site-packages/pluggy/_callers.py", line 103 in _multicall
  File "/home/sam/CLionProjects/numpy/venv/lib/python3.12/site-packages/pluggy/_manager.py", line 120 in _hookexec
  File "/home/sam/CLionProjects/numpy/venv/lib/python3.12/site-packages/pluggy/_hooks.py", line 513 in __call__
  File "/home/sam/CLionProjects/numpy/venv/lib/python3.12/site-packages/_pytest/python.py", line 1632 in runtest
  File "/home/sam/CLionProjects/numpy/venv/lib/python3.12/site-packages/_pytest/runner.py", line 173 in pytest_runtest_call
  File "/home/sam/CLionProjects/numpy/venv/lib/python3.12/site-packages/pluggy/_callers.py", line 103 in _multicall
  File "/home/sam/CLionProjects/numpy/venv/lib/python3.12/site-packages/pluggy/_manager.py", line 120 in _hookexec
  File "/home/sam/CLionProjects/numpy/venv/lib/python3.12/site-packages/pluggy/_hooks.py", line 513 in __call__
  File "/home/sam/CLionProjects/numpy/venv/lib/python3.12/site-packages/_pytest/runner.py", line 241 in <lambda>
  File "/home/sam/CLionProjects/numpy/venv/lib/python3.12/site-packages/_pytest/runner.py", line 341 in from_call
  File "/home/sam/CLionProjects/numpy/venv/lib/python3.12/site-packages/_pytest/runner.py", line 240 in call_and_report
  File "/home/sam/CLionProjects/numpy/venv/lib/python3.12/site-packages/_pytest/runner.py", line 135 in runtestprotocol
  File "/home/sam/CLionProjects/numpy/venv/lib/python3.12/site-packages/_pytest/runner.py", line 116 in pytest_runtest_protocol
  File "/home/sam/CLionProjects/numpy/venv/lib/python3.12/site-packages/pluggy/_callers.py", line 103 in _multicall
  File "/home/sam/CLionProjects/numpy/venv/lib/python3.12/site-packages/pluggy/_manager.py", line 120 in _hookexec
  File "/home/sam/CLionProjects/numpy/venv/lib/python3.12/site-packages/pluggy/_hooks.py", line 513 in __call__
  File "/home/sam/CLionProjects/numpy/venv/lib/python3.12/site-packages/_pytest/main.py", line 364 in pytest_runtestloop
  File "/home/sam/CLionProjects/numpy/venv/lib/python3.12/site-packages/pluggy/_callers.py", line 103 in _multicall
  File "/home/sam/CLionProjects/numpy/venv/lib/python3.12/site-packages/pluggy/_manager.py", line 120 in _hookexec
  File "/home/sam/CLionProjects/numpy/venv/lib/python3.12/site-packages/pluggy/_hooks.py", line 513 in __call__
  File "/home/sam/CLionProjects/numpy/venv/lib/python3.12/site-packages/_pytest/main.py", line 339 in _main
  File "/home/sam/CLionProjects/numpy/venv/lib/python3.12/site-packages/_pytest/main.py", line 285 in wrap_session
  File "/home/sam/CLionProjects/numpy/venv/lib/python3.12/site-packages/_pytest/main.py", line 332 in pytest_cmdline_main
  File "/home/sam/CLionProjects/numpy/venv/lib/python3.12/site-packages/pluggy/_callers.py", line 103 in _multicall
  File "/home/sam/CLionProjects/numpy/venv/lib/python3.12/site-packages/pluggy/_manager.py", line 120 in _hookexec
  File "/home/sam/CLionProjects/numpy/venv/lib/python3.12/site-packages/pluggy/_hooks.py", line 513 in __call__
  File "/home/sam/CLionProjects/numpy/venv/lib/python3.12/site-packages/_pytest/config/__init__.py", line 178 in main
  File "/home/sam/CLionProjects/numpy/venv/lib/python3.12/site-packages/_pytest/config/__init__.py", line 206 in console_main
  File "/home/sam/CLionProjects/numpy/venv/lib/python3.12/site-packages/pytest/__main__.py", line 7 in <module>
  File "<frozen runpy>", line 88 in _run_code
  File "<frozen runpy>", line 198 in _run_module_as_main

Extension modules: numpy._core._multiarray_umath, numpy.linalg._umath_linalg, numpy._core._multiarray_tests, numpy._core._rational_tests (total: 4)

Python and NumPy Versions:

Python 3.12.4 numpy main head: SHA256 2093a6d5b933f812d15a3de0eafeeb23c61f948a built with debug configuration, using MKL for blas/lapack (thought I don't know why that would make any difference).

Runtime Environment:

No response

Context for the issue:

No response

seberg commented 1 month ago

Maybe a stack overflow!? Since NumPy indicates recursion correctly, I am not sure why that would happen (Py 3.12 might have changes with recursion/stack overflow avoidance.).

But who knows, maybe there is also just something subtly wrong.

Can you put the test into a file.py:

import numpy as np

d = dict(names=['self'], formats=[None], offsets=[0])
d['formats'][0] = d
np.dtype(d)

and run it with spin gdb file.py. Then r (to run) and bt (backtrace) once it crashed and report the backtrace? Dunno if it will be helpful, but worth a shot (you need gdb installed of course).

mattip commented 1 week ago

Python makes a best effort to raise RecursionError before crashing, but sometimes the heuristics go awry. Debug builds mess with the heuristics, and I am not surprised it goes wrong somewhere. I will close this, please reopen or open a new issue if you have a fix we could do to the test to better catch the recursion.