P403n1x87
commented
11 months ago a core dump might be helpful if you could get one
Open juliangilbey opened 11 months ago
P403n1x87
commented
11 months ago a core dump might be helpful if you could get one
MatthieuDartiailh
commented
11 months ago Could you run the other tests by excluding the offending one to see if there are any other tests that trigger a similar behavior ?
juliangilbey
commented
11 months ago Thanks! It seems to be slightly inconsistent which test it crashes at (and occasionally it succeeds on the whole set of tests). (And it consistently succeeds with Python 3.11.) I've attached a core dump with the following Python traceback:
(sid_armhf-dchroot)jdg@abel:~/python-bytecode-0.15.1$ python3.12 -m pytest tests
============================= test session starts ==============================
platform linux -- Python 3.12.1, pytest-7.4.3, pluggy-1.3.0
rootdir: /home/jdg/python-bytecode-0.15.1
configfile: pyproject.toml
collected 165 items
tests/test_bytecode.py ...s...s..................sss..... [ 20%]
tests/test_cfg.py ........................Fatal Python error: Segmentation fault
Current thread 0xb6e28020 (most recent call first):
Garbage-collecting
File "/usr/lib/python3/dist-packages/bytecode/cfg.py", line 748 in from_bytecode
File "/usr/lib/python3/dist-packages/bytecode/bytecode.py", line 289 in compute_stacksize
File "/home/jdg/python-bytecode-0.15.1/tests/test_cfg.py", line 931 in test_huge_code_with_numerous_blocks
File "/usr/lib/python3.12/unittest/case.py", line 589 in _callTestMethod
File "/usr/lib/python3.12/unittest/case.py", line 636 in run
File "/usr/lib/python3.12/unittest/case.py", line 692 in __call__
File "/usr/lib/python3/dist-packages/_pytest/unittest.py", line 333 in runtest
File "/usr/lib/python3/dist-packages/_pytest/runner.py", line 169 in pytest_runtest_call
File "/usr/lib/python3/dist-packages/pluggy/_callers.py", line 77 in _multicall
File "/usr/lib/python3/dist-packages/pluggy/_manager.py", line 115 in _hookexec
File "/usr/lib/python3/dist-packages/pluggy/_hooks.py", line 493 in __call__
File "/usr/lib/python3/dist-packages/_pytest/runner.py", line 262 in <lambda>
File "/usr/lib/python3/dist-packages/_pytest/runner.py", line 341 in from_call
File "/usr/lib/python3/dist-packages/_pytest/runner.py", line 261 in call_runtest_hook
File "/usr/lib/python3/dist-packages/_pytest/runner.py", line 222 in call_and_report
File "/usr/lib/python3/dist-packages/_pytest/runner.py", line 133 in runtestprotocol
File "/usr/lib/python3/dist-packages/_pytest/runner.py", line 114 in pytest_runtest_protocol
File "/usr/lib/python3/dist-packages/pluggy/_callers.py", line 77 in _multicall
File "/usr/lib/python3/dist-packages/pluggy/_manager.py", line 115 in _hookexec
File "/usr/lib/python3/dist-packages/pluggy/_hooks.py", line 493 in __call__
File "/usr/lib/python3/dist-packages/_pytest/main.py", line 350 in pytest_runtestloop
File "/usr/lib/python3/dist-packages/pluggy/_callers.py", line 77 in _multicall
File "/usr/lib/python3/dist-packages/pluggy/_manager.py", line 115 in _hookexec
File "/usr/lib/python3/dist-packages/pluggy/_hooks.py", line 493 in __call__
File "/usr/lib/python3/dist-packages/_pytest/main.py", line 325 in _main
File "/usr/lib/python3/dist-packages/_pytest/main.py", line 271 in wrap_session
File "/usr/lib/python3/dist-packages/_pytest/main.py", line 318 in pytest_cmdline_main
File "/usr/lib/python3/dist-packages/pluggy/_callers.py", line 77 in _multicall
File "/usr/lib/python3/dist-packages/pluggy/_manager.py", line 115 in _hookexec
File "/usr/lib/python3/dist-packages/pluggy/_hooks.py", line 493 in __call__
File "/usr/lib/python3/dist-packages/_pytest/config/__init__.py", line 169 in main
File "/usr/lib/python3/dist-packages/_pytest/config/__init__.py", line 192 in console_main
File "/usr/lib/python3/dist-packages/pytest/__main__.py", line 5 in <module>
File "<frozen runpy>", line 88 in _run_code
File "<frozen runpy>", line 198 in _run_module_as_main
Segmentation fault (core dumped)Since I do not have a setup on which to easily open a core dump generated on ARM, could you reproduce the backtrace here ?
juliangilbey
commented
11 months ago Adding a bit more to my previous comment. I've just tried running `python3.12 with debug options:
(sid_armhf-dchroot)jdg@abel:~/python-bytecode-0.15.1$ python3.12 -d -X dev -m pytest tests
============================= test session starts ==============================
platform linux -- Python 3.12.1, pytest-7.4.3, pluggy-1.3.0
rootdir: /home/jdg/python-bytecode-0.15.1
configfile: pyproject.toml
collected 165 items
tests/test_bytecode.py ...s...s......Aborted (core dumped)Well, that's not very helpful!
And running gdb on the core dump attached in the previous comment and requesting a backtrace shows the following:
(sid_armhf-dchroot)jdg@abel:~/python-bytecode-0.15.1$ gdb /usr/bin/python3.12 ../core
[... intro lines snipped ...]
Reading symbols from /usr/bin/python3.12...
Reading symbols from /usr/lib/debug/.build-id/cb/7305157e9d17fb742d979b4a6c90bc2b135e7a.debug...
[New LWP 21180]
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/arm-linux-gnueabihf/libthread_db.so.1".
Core was generated by `python3.12 -m pytest tests'.
Program terminated with signal SIGSEGV, Segmentation fault.
#0 0xb6e61b06 in ?? () from /lib/arm-linux-gnueabihf/libc.so.6
(gdb) bt
#0 0xb6e61b06 in ?? () from /lib/arm-linux-gnueabihf/libc.so.6
#1 0xb6ea1084 in ?? () from /lib/arm-linux-gnueabihf/libc.so.6
#2 0xb6e70842 in raise () from /lib/arm-linux-gnueabihf/libc.so.6
#3 <signal handler called>
#4 0x000a12e6 in PyType_HasFeature (feature=<optimized out>,
type=<optimized out>) at ../Include/object.h:969
#5 _PyObject_IS_GC (obj=<unknown at remote 0xb6060011>)
at ../Include/internal/pycore_object.h:330
#6 visit_decref (parent=0xb60702e8, op=<unknown at remote 0xb6060011>)
at ../Modules/gcmodule.c:465
#7 list_traverse (o=0xb60702e8, visit=<optimized out>, arg=0xb60702e8)
at ../Objects/listobject.c:2705
#8 0x000a122a in subtract_refs (containers=0x54a108 <_PyRuntime+41656>)
at ../Modules/gcmodule.c:491
#9 deduce_unreachable (base=base@entry=0x54a108 <_PyRuntime+41656>,
unreachable=unreachable@entry=0xbe83a648) at ../Modules/gcmodule.c:1116
#10 0x000a0938 in gc_collect_main (
tstate=tstate@entry=0x5796d0 <_PyRuntime+235648>,
generation=generation@entry=2, n_collected=n_collected@entry=0xbe83a6ec,
n_uncollectable=n_uncollectable@entry=0xbe83a6e8, nofail=<optimized out>,
nofail@entry=0) at ../Modules/gcmodule.c:1242
#11 0x00160a80 in gc_collect_with_callback (
tstate=0x5796d0 <_PyRuntime+235648>, generation=2)
at ../Modules/gcmodule.c:1426
#12 0x00160a2c in gc_collect_generations.isra.0 (
tstate=tstate@entry=0x5796d0 <_PyRuntime+235648>)
at ../Modules/gcmodule.c:1481
#13 0x0009a5c6 in _Py_RunGC (tstate=0x5796d0 <_PyRuntime+235648>)
at ../Modules/gcmodule.c:2292
#14 _Py_HandlePending (tstate=<optimized out>) at ../Python/ceval_gil.c:1045
#15 _PyEval_EvalFrameDefault (tstate=<optimized out>, frame=<optimized out>,
throwflag=<optimized out>) at ../Python/ceval.c:834
#16 0x000e6318 in _PyEval_EvalFrame (throwflag=0, frame=0xb6d45ac4,
tstate=0x5796d0 <_PyRuntime+235648>)
at ../Include/internal/pycore_ceval.h:89
#17 _PyEval_Vector (kwnames=('result',), argcount=1, args=0xb6d5d670,
locals=0x0, func=0xb66ec758, tstate=0x5796d0 <_PyRuntime+235648>)
at ../Python/ceval.c:1683
#18 _PyFunction_Vectorcall (kwnames=('result',), nargsf=<optimized out>,
stack=0xb6d5d670, func=<function at remote 0xb66ec758>)
at ../Objects/call.c:419
#19 _PyObject_VectorcallTstate (tstate=0x5796d0 <_PyRuntime+235648>,
callable=<function at remote 0xb66ec758>, args=0xb6d5d670,
nargsf=<optimized out>, kwnames=<optimized out>)
at ../Include/internal/pycore_call.h:92
#20 0x000e60f4 in method_vectorcall (method=<optimized out>, args=0xb6d5d674,
nargsf=<optimized out>, kwnames=('result',)) at ../Objects/classobject.c:61
#21 0x000c9194 in _PyVectorcall_Call (kwargs=<optimized out>,
tuple=<optimized out>, callable=<method at remote 0xb5fc77c8>,
func=0xe6061 <method_vectorcall>, tstate=0x5796d0 <_PyRuntime+235648>)
at ../Objects/call.c:283
#22 _PyObject_Call (tstate=0x5796d0 <_PyRuntime+235648>,
callable=<method at remote 0xb5fc77c8>, args=<optimized out>,
kwargs=<optimized out>) at ../Objects/call.c:354
#23 0x000950a6 in PyCFunction_Call (
kwargs={'result': <TestCaseFunction(keywords=<NodeKeywords at remote 0xb60b31e8>, own_markers=[], extra_keyword_matches=set(), stash=<Stash at remote 0xb60a9eb0>, _report_sections=[], user_properties=[], _obj=<method at remote 0xb5fc7228>, originalname='test_huge_code_with_numerous_blocks', _fixtureinfo=<FuncFixtureInfo at remote 0xb60b3268>, fixturenames=['_unittest_setUpClass_fixture_CFGStacksizeComputationTests', 'request'], funcargs={'_unittest_setUpClass_fixture_CFGStacksizeComputationTests': None, 'request': <FixtureRequest(_pyfuncitem=<...>, fixturename=None, _scope=<Scope(_value_='function', _name_='Function', __objclass__=<EnumType(_generate_next_value_=<staticmethod at remote 0xb679bfa0>, __module__='_pytest.scope', __annotations__={'Function': '_ScopeName', 'Class': '_ScopeName', 'Module': '_ScopeName', 'Package': '_ScopeName', 'Session': '_ScopeName'}, __doc__='\n Represents one of the possible fixture scopes in pytest.\n\n Scopes are ordered from lower to higher, that is:\n\n ->>> hig...(truncated), args=(), callable=<method at remote 0xb5fc77c8>)
at ../Objects/call.c:387
#24 _PyEval_EvalFrameDefault (tstate=<optimized out>, frame=<optimized out>,
throwflag=<optimized out>) at Python/bytecodes.c:3254
#25 0x00088f4a in _PyObject_FastCallDictTstate (
tstate=0x5796d0 <_PyRuntime+235648>,
callable=<function at remote 0xb66ec848>, args=0xbe83a98c,
nargsf=<optimized out>, kwargs=<optimized out>) at ../Objects/call.c:144
#26 0x000c64cc in _PyObject_Call_Prepend (tstate=0x5796d0 <_PyRuntime+235648>,
callable=<function at remote 0xb66ec848>,
obj=<CFGStacksizeComputationTests(_testMethodName='test_huge_code_with_numerous_blocks', _outcome=<_Outcome(expecting_failure=False, result=<TestCaseFunction(keywords=<NodeKeywords at remote 0xb60b31e8>, own_markers=[], extra_keyword_matches=set(), stash=<Stash at remote 0xb60a9eb0>, _report_sections=[], user_properties=[], _obj=<method at remote 0xb5fc7228>, originalname='test_huge_code_with_numerous_blocks', _fixtureinfo=<FuncFixtureInfo at remote 0xb60b3268>, fixturenames=['_unittest_setUpClass_fixture_CFGStacksizeComputationTests', 'request'], funcargs={'_unittest_setUpClass_fixture_CFGStacksizeComputationTests': None, 'request': <FixtureRequest(_pyfuncitem=<...>, fixturename=None, _scope=<Scope(_value_='function', _name_='Function', __objclass__=<EnumType(_generate_next_value_=<staticmethod at remote 0xb679bfa0>, __module__='_pytest.scope', __annotations__={'Function': '_ScopeName', 'Class': '_ScopeName', 'Module': '_ScopeName', 'Package': '_ScopeName', 'Session': '_ScopeName'}, __doc__='\n Represents one...(truncated), args=<optimized out>, kwargs=<optimized out>) at ../Objects/call.c:508
#27 0x001acd84 in slot_tp_call (
self=<CFGStacksizeComputationTests(_testMethodName='test_huge_code_with_numerous_blocks', _outcome=<_Outcome(expecting_failure=False, result=<TestCaseFunction(keywords=<NodeKeywords at remote 0xb60b31e8>, own_markers=[], extra_keyword_matches=set(), stash=<Stash at remote 0xb60a9eb0>, _report_sections=[], user_properties=[], _obj=<method at remote 0xb5fc7228>, originalname='test_huge_code_with_numerous_blocks', _fixtureinfo=<FuncFixtureInfo at remote 0xb60b3268>, fixturenames=['_unittest_setUpClass_fixture_CFGStacksizeComputationTests', 'request'], funcargs={'_unittest_setUpClass_fixture_CFGStacksizeComputationTests': None, 'request': <FixtureRequest(_pyfuncitem=<...>, fixturename=None, _scope=<Scope(_value_='function', _name_='Function', __objclass__=<EnumType(_generate_next_value_=<staticmethod at remote 0xb679bfa0>, __module__='_pytest.scope', __annotations__={'Function': '_ScopeName', 'Class': '_ScopeName', 'Module': '_ScopeName', 'Package': '_ScopeName', 'Session': '_ScopeName'}, __doc__='\n Represents one...(truncated), args=(),
kwds={'result': <TestCaseFunction(keywords=<NodeKeywords at remote 0xb60b31e8>, own_markers=[], extra_keyword_matches=set(), stash=<Stash at remote 0xb60a9eb0>, _report_sections=[], user_properties=[], _obj=<method at remote 0xb5fc7228>, originalname='test_huge_code_with_numerous_blocks', _fixtureinfo=<FuncFixtureInfo at remote 0xb60b3268>, fixturenames=['_unittest_setUpClass_fixture_CFGStacksizeComputationTests', 'request'], funcargs={'_unittest_setUpClass_fixture_CFGStacksizeComputationTests': None, 'request': <FixtureRequest(_pyfuncitem=<...>, fixturename=None, _scope=<Scope(_value_='function', _name_='Function', __objclass__=<EnumType(_generate_next_value_=<staticmethod at remote 0xb679bfa0>, __module__='_pytest.scope', __annotations__={'Function': '_ScopeName', 'Class': '_ScopeName', 'Module': '_ScopeName', 'Package': '_ScopeName', 'Session': '_ScopeName'}, __doc__='\n Represents one of the possible fixture scopes in pytest.\n\n Scopes are ordered from lower to higher, that is:\n\n ->>> hig...(truncated)) at ../Objects/typeobject.c:8769
#28 0x00083394 in _PyObject_MakeTpCall (tstate=0x5796d0 <_PyRuntime+235648>,
callable=<CFGStacksizeComputationTests(_testMethodName='test_huge_code_with_numerous_blocks', _outcome=<_Outcome(expecting_failure=False, result=<TestCaseFunction(keywords=<NodeKeywords at remote 0xb60b31e8>, own_markers=[], extra_keyword_matches=set(), stash=<Stash at remote 0xb60a9eb0>, _report_sections=[], user_properties=[], _obj=<method at remote 0xb5fc7228>, originalname='test_huge_code_with_numerous_blocks', _fixtureinfo=<FuncFixtureInfo at remote 0xb60b3268>, fixturenames=['_unittest_setUpClass_fixture_CFGStacksizeComputationTests', 'request'], funcargs={'_unittest_setUpClass_fixture_CFGStacksizeComputationTests': None, 'request': <FixtureRequest(_pyfuncitem=<...>, fixturename=None, _scope=<Scope(_value_='function', _name_='Function', __objclass__=<EnumType(_generate_next_value_=<staticmethod at remote 0xb679bfa0>, __module__='_pytest.scope', __annotations__={'Function': '_ScopeName', 'Class': '_ScopeName', 'Module': '_ScopeName', 'Package': '_ScopeName', 'Session': '_ScopeName'}, __doc__='\n Represents one...(truncated), args=<optimized out>, nargs=0, keywords=('result',))
at ../Objects/call.c:240
#29 0x000a8980 in _PyObject_VectorcallTstate (kwnames=('result',),
nargsf=<optimized out>, args=<optimized out>,
callable=<CFGStacksizeComputationTests(_testMethodName='test_huge_code_with_numerous_blocks', _outcome=<_Outcome(expecting_failure=False, result=<TestCaseFunction(keywords=<NodeKeywords at remote 0xb60b31e8>, own_markers=[], extra_keyword_matches=set(), stash=<Stash at remote 0xb60a9eb0>, _report_sections=[], user_properties=[], _obj=<method at remote 0xb5fc7228>, originalname='test_huge_code_with_numerous_blocks', _fixtureinfo=<FuncFixtureInfo at remote 0xb60b3268>, fixturenames=['_unittest_setUpClass_fixture_CFGStacksizeComputationTests', 'request'], funcargs={'_unittest_setUpClass_fixture_CFGStacksizeComputationTests': None, 'request': <FixtureRequest(_pyfuncitem=<...>, fixturename=None, _scope=<Scope(_value_='function', _name_='Function', __objclass__=<EnumType(_generate_next_value_=<staticmethod at remote 0xb679bfa0>, __module__='_pytest.scope', __annotations__={'Function': '_ScopeName', 'Class': '_ScopeName', 'Module': '_ScopeName', 'Package': '_ScopeName', 'Session': '_ScopeName'}, __doc__='\n Represents one...(truncated), tstate=0x5796d0 <_PyRuntime+235648>)
at ../Include/internal/pycore_call.h:90
#30 _PyObject_VectorcallTstate (kwnames=('result',), nargsf=<optimized out>,
args=<optimized out>,
callable=<CFGStacksizeComputationTests(_testMethodName='test_huge_code_with_numerous_blocks', _outcome=<_Outcome(expecting_failure=False, result=<TestCaseFunction(keywords=<NodeKeywords at remote 0xb60b31e8>, own_markers=[], extra_keyword_matches=set(), stash=<Stash at remote 0xb60a9eb0>, _report_sections=[], user_properties=[], _obj=<method at remote 0xb5fc7228>, originalname='test_huge_code_with_numerous_blocks', _fixtureinfo=<FuncFixtureInfo at remote 0xb60b3268>, fixturenames=['_unittest_setUpClass_fixture_CFGStacksizeComputationTests', 'request'], funcargs={'_unittest_setUpClass_fixture_CFGStacksizeComputationTests': None, 'request': <FixtureRequest(_pyfuncitem=<...>, fixturename=None, _scope=<Scope(_value_='function', _name_='Function', __objclass__=<EnumType(_generate_next_value_=<staticmethod at remote 0xb679bfa0>, __module__='_pytest.scope', __annotations__={'Function': '_ScopeName', 'Class': '_ScopeName', 'Module': '_ScopeName', 'Package': '_ScopeName', 'Session': '_ScopeName'}, __doc__='\n Represents one...(truncated), tstate=0x5796d0 <_PyRuntime+235648>)
at ../Include/internal/pycore_call.h:77
#31 PyObject_Vectorcall (
callable=<CFGStacksizeComputationTests(_testMethodName='test_huge_code_with_numerous_blocks', _outcome=<_Outcome(expecting_failure=False, result=<TestCaseFunction(keywords=<NodeKeywords at remote 0xb60b31e8>, own_markers=[], extra_keyword_matches=set(), stash=<Stash at remote 0xb60a9eb0>, _report_sections=[], user_properties=[], _obj=<method at remote 0xb5fc7228>, originalname='test_huge_code_with_numerous_blocks', _fixtureinfo=<FuncFixtureInfo at remote 0xb60b3268>, fixturenames=['_unittest_setUpClass_fixture_CFGStacksizeComputationTests', 'request'], funcargs={'_unittest_setUpClass_fixture_CFGStacksizeComputationTests': None, 'request': <FixtureRequest(_pyfuncitem=<...>, fixturename=None, _scope=<Scope(_value_='function', _name_='Function', __objclass__=<EnumType(_generate_next_value_=<staticmethod at remote 0xb679bfa0>, __module__='_pytest.scope', __annotations__={'Function': '_ScopeName', 'Class': '_ScopeName', 'Module': '_ScopeName', 'Package': '_ScopeName', 'Session': '_ScopeName'}, __doc__='\n Represents one...(truncated), args=<optimized out>, nargsf=<optimized out>, kwnames=('result',))
at ../Objects/call.c:325
#32 0x00092334 in _PyEval_EvalFrameDefault (tstate=<optimized out>,
frame=<optimized out>, throwflag=<optimized out>)
at Python/bytecodes.c:2706
#33 0x00088f4a in _PyObject_FastCallDictTstate (
tstate=0x5796d0 <_PyRuntime+235648>,
callable=<function at remote 0xb69307f8>, args=0xbe83ab4c,
nargsf=<optimized out>, kwargs=<optimized out>) at ../Objects/call.c:144
#34 0x000c64cc in _PyObject_Call_Prepend (tstate=0x5796d0 <_PyRuntime+235648>,
callable=<function at remote 0xb69307f8>,
obj=<HookCaller at remote 0xb6627bb8>, args=<optimized out>,
kwargs=<optimized out>) at ../Objects/call.c:508
#35 0x001acd84 in slot_tp_call (self=<HookCaller at remote 0xb6627bb8>,
args=(),
kwds={'item': <TestCaseFunction(keywords=<NodeKeywords at remote 0xb60b31e8>, own_markers=[], extra_keyword_matches=set(), stash=<Stash at remote 0xb60a9eb0>, _report_sections=[], user_properties=[], _obj=<method at remote 0xb5fc7228>, originalname='test_huge_code_with_numerous_blocks', _fixtureinfo=<FuncFixtureInfo at remote 0xb60b3268>, fixturenames=['_unittest_setUpClass_fixture_CFGStacksizeComputationTests', 'request'], funcargs={'_unittest_setUpClass_fixture_CFGStacksizeComputationTests': None, 'request': <FixtureRequest(_pyfuncitem=<...>, fixturename=None, _scope=<Scope(_value_='function', _name_='Function', __objclass__=<EnumType(_generate_next_value_=<staticmethod at remote 0xb679bfa0>, __module__='_pytest.scope', __annotations__={'Function': '_ScopeName', 'Class': '_ScopeName', 'Module': '_ScopeName', 'Package': '_ScopeName', 'Session': '_ScopeName'}, __doc__='\n Represents one of the possible fixture scopes in pytest.\n\n Scopes are ordered from lower to higher, that is:\n\n ->>> highe...(truncated)) at ../Objects/typeobject.c:8769
#36 0x000c913a in _PyObject_Call (tstate=0x5796d0 <_PyRuntime+235648>,
callable=<HookCaller at remote 0xb6627bb8>, args=(),
kwargs=<optimized out>) at ../Objects/call.c:367
#37 0x000950a6 in PyCFunction_Call (
kwargs={'item': <TestCaseFunction(keywords=<NodeKeywords at remote 0xb60b31e8>, own_markers=[], extra_keyword_matches=set(), stash=<Stash at remote 0xb60a9eb0>, _report_sections=[], user_properties=[], _obj=<method at remote 0xb5fc7228>, originalname='test_huge_code_with_numerous_blocks', _fixtureinfo=<FuncFixtureInfo at remote 0xb60b3268>, fixturenames=['_unittest_setUpClass_fixture_CFGStacksizeComputationTests', 'request'], funcargs={'_unittest_setUpClass_fixture_CFGStacksizeComputationTests': None, 'request': <FixtureRequest(_pyfuncitem=<...>, fixturename=None, _scope=<Scope(_value_='function', _name_='Function', __objclass__=<EnumType(_generate_next_value_=<staticmethod at remote 0xb679bfa0>, __module__='_pytest.scope', __annotations__={'Function': '_ScopeName', 'Class': '_ScopeName', 'Module': '_ScopeName', 'Package': '_ScopeName', 'Session': '_ScopeName'}, __doc__='\n Represents one of the possible fixture scopes in pytest.\n\n Scopes are ordered from lower to higher, that is:\n\n ->>> highe...(truncated), args=(), callable=<HookCaller at remote 0xb6627bb8>)
at ../Objects/call.c:387
#38 _PyEval_EvalFrameDefault (tstate=<optimized out>, frame=<optimized out>,
throwflag=<optimized out>) at Python/bytecodes.c:3254
#39 0x00088f4a in _PyObject_FastCallDictTstate (
tstate=0x5796d0 <_PyRuntime+235648>,
callable=<function at remote 0xb69307f8>, args=0xbe83acf4,
nargsf=<optimized out>, kwargs=<optimized out>) at ../Objects/call.c:144
#40 0x000c64cc in _PyObject_Call_Prepend (tstate=0x5796d0 <_PyRuntime+235648>,
callable=<function at remote 0xb69307f8>,
obj=<HookCaller at remote 0xb6627de8>, args=<optimized out>,
kwargs=<optimized out>) at ../Objects/call.c:508
#41 0x001acd84 in slot_tp_call (self=<HookCaller at remote 0xb6627de8>,
args=(),
kwds={'item': <TestCaseFunction(keywords=<NodeKeywords at remote 0xb60b31e8>, own_markers=[], extra_keyword_matches=set(), stash=<Stash at remote 0xb60a9eb0>, _report_sections=[], user_properties=[], _obj=<method at remote 0xb5fc7228>, originalname='test_huge_code_with_numerous_blocks', _fixtureinfo=<FuncFixtureInfo at remote 0xb60b3268>, fixturenames=['_unittest_setUpClass_fixture_CFGStacksizeComputationTests', 'request'], funcargs={'_unittest_setUpClass_fixture_CFGStacksizeComputationTests': None, 'request': <FixtureRequest(_pyfuncitem=<...>, fixturename=None, _scope=<Scope(_value_='function', _name_='Function', __objclass__=<EnumType(_generate_next_value_=<staticmethod at remote 0xb679bfa0>, __module__='_pytest.scope', __annotations__={'Function': '_ScopeName', 'Class': '_ScopeName', 'Module': '_ScopeName', 'Package': '_ScopeName', 'Session': '_ScopeName'}, __doc__='\n Represents one of the possible fixture scopes in pytest.\n\n Scopes are ordered from lower to higher, that is:\n\n ->>> highe...(truncated)) at ../Objects/typeobject.c:8769
#42 0x00083394 in _PyObject_MakeTpCall (tstate=0x5796d0 <_PyRuntime+235648>,
callable=<HookCaller at remote 0xb6627de8>, args=<optimized out>, nargs=0,
keywords=('item', 'nextitem')) at ../Objects/call.c:240
#43 0x000a8980 in _PyObject_VectorcallTstate (kwnames=('item', 'nextitem'),
nargsf=<optimized out>, args=<optimized out>,
callable=<HookCaller at remote 0xb6627de8>,
tstate=0x5796d0 <_PyRuntime+235648>)
at ../Include/internal/pycore_call.h:90
#44 _PyObject_VectorcallTstate (kwnames=('item', 'nextitem'),
nargsf=<optimized out>, args=<optimized out>,
callable=<HookCaller at remote 0xb6627de8>,
tstate=0x5796d0 <_PyRuntime+235648>)
at ../Include/internal/pycore_call.h:77
#45 PyObject_Vectorcall (callable=<HookCaller at remote 0xb6627de8>,
args=<optimized out>, nargsf=<optimized out>, kwnames=('item', 'nextitem'))
at ../Objects/call.c:325
#46 0x00092334 in _PyEval_EvalFrameDefault (tstate=<optimized out>,
frame=<optimized out>, throwflag=<optimized out>)
at Python/bytecodes.c:2706
#47 0x00088f4a in _PyObject_FastCallDictTstate (
tstate=0x5796d0 <_PyRuntime+235648>,
callable=<function at remote 0xb69307f8>, args=0xbe83aeb4,
nargsf=<optimized out>, kwargs=<optimized out>) at ../Objects/call.c:144
#48 0x000c64cc in _PyObject_Call_Prepend (tstate=0x5796d0 <_PyRuntime+235648>,
callable=<function at remote 0xb69307f8>,
obj=<HookCaller at remote 0xb6627f00>, args=<optimized out>,
kwargs=<optimized out>) at ../Objects/call.c:508
#49 0x001acd84 in slot_tp_call (self=<HookCaller at remote 0xb6627f00>,
args=(),
kwds={'session': <Session(keywords=<NodeKeywords at remote 0xb658eb08>, own_markers=[], extra_keyword_matches=set(), stash=<Stash at remote 0xb6621d78>, testsfailed=0, testscollected=165, shouldstop=False, shouldfail=False, trace=<TagTracerSub(root=<TagTracer(_tags2proc={}, _writer=None, indent=0) at remote 0xb6d86e58>, tags=('collection',)) at remote 0xb66214b0>, _initialpaths=frozenset({<PosixPath at remote 0xb655b840>}), _bestrelpathcache=<_bestrelpath_cache at remote 0xb67a2028>, exitstatus=<ExitCode(_value_=0, _name_='OK', __objclass__=<EnumType(_generate_next_value_=<staticmethod at remote 0xb68503e8>, __module__='_pytest.config', __doc__='Encodes the valid exit codes by pytest.\n\n Currently users and plugins may supply other exit codes as well.\n\n .. versionadded:: 5.0\n ', _new_member_=<built-in method __new__ of type object at remote 0x4a2554>, _use_args_=True, _member_names_=['OK', 'TESTS_FAILED', 'INTERRUPTED', 'INTERNAL_ERROR', 'USAGE_ERROR', 'NO_TESTS_COLLECTED'], _member_map_={'OK': <...>,...(truncated)) at ../Objects/typeobject.c:8769
#50 0x00083394 in _PyObject_MakeTpCall (tstate=0x5796d0 <_PyRuntime+235648>,
callable=<HookCaller at remote 0xb6627f00>, args=<optimized out>, nargs=0,
keywords=('session',)) at ../Objects/call.c:240
#51 0x000a8980 in _PyObject_VectorcallTstate (kwnames=('session',),
nargsf=<optimized out>, args=<optimized out>,
callable=<HookCaller at remote 0xb6627f00>,
tstate=0x5796d0 <_PyRuntime+235648>)
at ../Include/internal/pycore_call.h:90
#52 _PyObject_VectorcallTstate (kwnames=('session',), nargsf=<optimized out>,
args=<optimized out>, callable=<HookCaller at remote 0xb6627f00>,
tstate=0x5796d0 <_PyRuntime+235648>)
at ../Include/internal/pycore_call.h:77
#53 PyObject_Vectorcall (callable=<HookCaller at remote 0xb6627f00>,
args=<optimized out>, nargsf=<optimized out>, kwnames=('session',))
at ../Objects/call.c:325
#54 0x00092334 in _PyEval_EvalFrameDefault (tstate=<optimized out>,
frame=<optimized out>, throwflag=<optimized out>)
at Python/bytecodes.c:2706
#55 0x00088f4a in _PyObject_FastCallDictTstate (
tstate=0x5796d0 <_PyRuntime+235648>,
callable=<function at remote 0xb69307f8>, args=0xbe83b074,
nargsf=<optimized out>, kwargs=<optimized out>) at ../Objects/call.c:144
#56 0x000c64cc in _PyObject_Call_Prepend (tstate=0x5796d0 <_PyRuntime+235648>,
callable=<function at remote 0xb69307f8>,
obj=<HookCaller at remote 0xb661fd98>, args=<optimized out>,
kwargs=<optimized out>) at ../Objects/call.c:508
#57 0x001acd84 in slot_tp_call (self=<HookCaller at remote 0xb661fd98>,
args=(),
kwds={'config': <Config(option=<Namespace(keyword='', markexpr='', maxfail=0, continue_on_collection_errors=False, confcutdir=None, noconftest=False, keepduplicates=False, collect_in_virtualenv=False, importmode='prepend', basetemp=None, durations=None, durations_min=<float at remote 0xb6936290>, version=0, plugins=[], traceconfig=False, showfixtures=False, show_fixtures_per_test=False, verbose=0, no_header=False, no_summary=False, reportchars='fE', disable_warnings=False, showlocals=False, tbstyle='auto', showcapture='all', fulltrace=False, color='auto', code_highlight='yes', capture='fd', runxfail=False, pastebin=None, assertmode='rewrite', xmlpath=None, junitprefix=None, doctestmodules=False, doctestreport='udiff', doctestglob=[], doctest_ignore_import_errors=False, doctest_continue_on_failure=False, last_failed_no_failures='all', stepwise=False, stepwise_skip=False, logger_disable=[], markers=False, usepdb=False, usepdb_cls=None, trace=False, lf=False, failedfirst=False, newfirst=False, cacheshow=None, cachecl...(truncated)) at ../Objects/typeobject.c:8769
#58 0x00083394 in _PyObject_MakeTpCall (tstate=0x5796d0 <_PyRuntime+235648>,
callable=<HookCaller at remote 0xb661fd98>, args=<optimized out>, nargs=0,
keywords=('config',)) at ../Objects/call.c:240
#59 0x000a8980 in _PyObject_VectorcallTstate (kwnames=('config',),
nargsf=<optimized out>, args=<optimized out>,
callable=<HookCaller at remote 0xb661fd98>,
tstate=0x5796d0 <_PyRuntime+235648>)
at ../Include/internal/pycore_call.h:90
#60 _PyObject_VectorcallTstate (kwnames=('config',), nargsf=<optimized out>,
args=<optimized out>, callable=<HookCaller at remote 0xb661fd98>,
tstate=0x5796d0 <_PyRuntime+235648>)
at ../Include/internal/pycore_call.h:77
#61 PyObject_Vectorcall (callable=<HookCaller at remote 0xb661fd98>,
args=<optimized out>, nargsf=<optimized out>, kwnames=('config',))
at ../Objects/call.c:325
#62 0x00092334 in _PyEval_EvalFrameDefault (tstate=<optimized out>,
frame=<optimized out>, throwflag=<optimized out>)
at Python/bytecodes.c:2706
#63 0x0015fda6 in _PyEval_EvalFrame (throwflag=0, frame=0xb6d450e4,
tstate=0x5796d0 <_PyRuntime+235648>)
at ../Include/internal/pycore_ceval.h:89
#64 _PyEval_Vector (args=0x0, argcount=0, kwnames=0x0, locals=<optimized out>,
func=0xb6d5ae88, tstate=0x5796d0 <_PyRuntime+235648>)
at ../Python/ceval.c:1683
#65 PyEval_EvalCode (co=<optimized out>, globals=<optimized out>,
locals=<optimized out>) at ../Python/ceval.c:578
#66 0x00176828 in builtin_exec_impl (module=<optimized out>,
closure=<optimized out>,
locals={'__name__': '__main__', '__doc__': 'The pytest entry point.', '__package__': 'pytest', '__loader__': <SourceFileLoader(name='pytest.__main__', path='/usr/lib/python3/dist-packages/pytest/__main__.py') at remote 0xb670d348>, '__spec__': <ModuleSpec(name='pytest.__main__', loader=<...>, origin='/usr/lib/python3/dist-packages/pytest/__main__.py', loader_state=None, submodule_search_locations=None, _uninitialized_submodules=[], _set_fileattr=True, _cached='/usr/lib/python3/dist-packages/pytest/__pycache__/__main__.cpython-312.pyc') at remote 0xb6cc66a8>, '__annotations__': {}, '__builtins__': <module at remote 0xb6d53398>, '__file__': '/usr/lib/python3/dist-packages/pytest/__main__.py', '__cached__': '/usr/lib/python3/dist-packages/pytest/__pycache__/__main__.cpython-312.pyc', 'pytest': <module at remote 0xb6d408c0>},
globals={'__name__': '__main__', '__doc__': 'The pytest entry point.', '__package__': 'pytest', '__loader__': <SourceFileLoader(name='pytest.__main__', path='/usr/lib/python3/dist-packages/pytest/__main__.py') at remote 0xb670d348>, '__spec__': <ModuleSpec(name='pytest.__main__', loader=<...>, origin='/usr/lib/python3/dist-packages/pytest/__main__.py', loader_state=None, submodule_search_locations=None, _uninitialized_submodules=[], _set_fileattr=True, _cached='/usr/lib/python3/dist-packages/pytest/__pycache__/__main__.cpython-312.pyc') at remote 0xb6cc66a8>, '__annotations__': {}, '__builtins__': <module at remote 0xb6d53398>, '__file__': '/usr/lib/python3/dist-packages/pytest/__main__.py', '__cached__': '/usr/lib/python3/dist-packages/pytest/__pycache__/__main__.cpython-312.pyc', 'pytest': <module at remote 0xb6d408c0>}, source=<code at remote 0xb662ea48>)
at ../Python/bltinmodule.c:1096
#67 builtin_exec (module=<optimized out>, args=<optimized out>,
nargs=<optimized out>, kwnames=<optimized out>)
at ../Python/clinic/bltinmodule.c.h:586
#68 0x000a8b48 in cfunction_vectorcall_FASTCALL_KEYWORDS (
func=<built-in method exec of module object at remote 0xb6d53398>,
args=0xb6d450c8, nargsf=<optimized out>, kwnames=<optimized out>)
at ../Objects/methodobject.c:438
#69 0x000a895e in _PyObject_VectorcallTstate (kwnames=0x0,
nargsf=<optimized out>, args=<optimized out>,
callable=<built-in method exec of module object at remote 0xb6d53398>,
tstate=0x5796d0 <_PyRuntime+235648>)
at ../Include/internal/pycore_call.h:92
#70 PyObject_Vectorcall (
callable=<built-in method exec of module object at remote 0xb6d53398>,
args=<optimized out>, nargsf=<optimized out>, kwnames=0x0)
at ../Objects/call.c:325
#71 0x00092334 in _PyEval_EvalFrameDefault (tstate=<optimized out>,
frame=<optimized out>, throwflag=<optimized out>)
at Python/bytecodes.c:2706
#72 0x0018ae02 in pymain_run_module (modname=<optimized out>, set_argv0=1)
at ../Modules/main.c:300
#73 0x0018a38c in pymain_run_python (exitcode=0xbe83b3c0)
at ../Modules/main.c:623
#74 Py_RunMain () at ../Modules/main.c:709
#75 0x0014ea00 in Py_BytesMain (argc=4, argv=<optimized out>)
at ../Modules/main.c:763
#76 0xb6e617da in ?? () from /lib/arm-linux-gnueabihf/libc.so.6
#77 0xb6e6187e in __libc_start_main () from /lib/arm-linux-gnueabihf/libc.so.6
#78 0x0014e8e4 in _start ()
Backtrace stopped: previous frame identical to this frame (corrupt stack?)
(gdb) I don't know much at all about Python's internals, but my limited understanding from this backtrace is that it's crashed inside the garbage collector (#10), though I may be completely wrong about this.
P403n1x87
commented
11 months ago To me the most obvious cause is a potential CPython frame stack corruption (e.g. a missing RESUME or other prefix instructions etc...). Tests crashing only with CPython >= 3.11 might be an indication of this.
juliangilbey
commented
11 months ago I tried running with the failing test excluded:
(sid_armhf-dchroot)jdg@abel:~/python-bytecode-0.15.1$ python3.12 -m pytest tests -k "not test_huge_code_with_numerous_blocks"
============================= test session starts ==============================
platform linux -- Python 3.12.1, pytest-7.4.3, pluggy-1.3.0
rootdir: /home/jdg/python-bytecode-0.15.1
configfile: pyproject.toml
collected 165 items / 1 deselected / 164 selected
tests/test_bytecode.py ...s...s..................sss..... [ 20%]
tests/test_cfg.py ......................................... [ 45%]
tests/test_code.py ...... [ 49%]
tests/test_concrete.py ......................s.......s........Fatal Python error: Segmentation fault
Current thread 0xb6da4020 (most recent call first):
Garbage-collecting
File "/usr/lib/python3/dist-packages/bytecode/instr.py", line 528 in __init__
File "/usr/lib/python3/dist-packages/bytecode/concrete.py", line 1068 in to_bytecode
File "/usr/lib/python3/dist-packages/bytecode/concrete.py", line 862 in to_code
File "/home/jdg/python-bytecode-0.15.1/tests/test_concrete.py", line 1514 in test_extended_jump
File "/usr/lib/python3.12/unittest/case.py", line 589 in _callTestMethod
[...]Trying to exclude both of these tests was even less informative:
(sid_armhf-dchroot)jdg@abel:~/python-bytecode-0.15.1$ python3.12 -m pytest tests -k "not test_huge_code_with_numerous_blocks and not test_extended_jump"
============================= test session starts ==============================
platform linux -- Python 3.12.1, pytest-7.4.3, pluggy-1.3.0
rootdir: /home/jdg/python-bytecode-0.15.1
configfile: pyproject.toml
collected 165 items / 2 deselected / 163 selected
tests/test_bytecode.py ...s...s..................sss..... [ 20%]
tests/test_cfg.py ......................................... [ 46%]
tests/test_code.py ...... [ 49%]
tests/test_concrete.py ......................s.......s............... [ 77%]
tests/test_flags.py ..ss.. [ 81%]
tests/test_instr.py ........................ [ 96%]
tests/test_misc.py ...... [100%]
================ 154 passed, 9 skipped, 2 deselected in 17.94s =================
Segmentation faultLooking at the core dump in this last case, which is very short, it again fails in the garbage collection:
Core was generated by `python3.12 -m pytest tests -k not test_huge_code_with_numerous_blocks and not t'.
Program terminated with signal SIGSEGV, Segmentation fault.
#0 0x000a12e6 in PyType_HasFeature (feature=<optimized out>,
type=<optimized out>) at ../Include/object.h:969
969 ../Include/object.h: No such file or directory.
(gdb) bt
#0 0x000a12e6 in PyType_HasFeature (feature=<optimized out>,
type=<optimized out>) at ../Include/object.h:969
#1 _PyObject_IS_GC (obj=<unknown at remote 0xb6060011>)
at ../Include/internal/pycore_object.h:330
#2 visit_decref (parent=0xb60633a8, op=<unknown at remote 0xb6060011>)
at ../Modules/gcmodule.c:465
#3 list_traverse (o=0xb60633a8, visit=<optimized out>, arg=0xb60633a8)
at ../Objects/listobject.c:2705
#4 0x000a122a in subtract_refs (containers=0x54a108 <_PyRuntime+41656>)
at ../Modules/gcmodule.c:491
#5 deduce_unreachable (base=base@entry=0x54a108 <_PyRuntime+41656>,
unreachable=unreachable@entry=0xbeabd250) at ../Modules/gcmodule.c:1116
#6 0x000a0938 in gc_collect_main (
tstate=tstate@entry=0x5796d0 <_PyRuntime+235648>,
generation=generation@entry=2, n_collected=n_collected@entry=0xbeabd2f4,
n_uncollectable=n_uncollectable@entry=0xbeabd2f0, nofail=<optimized out>,
nofail@entry=0) at ../Modules/gcmodule.c:1242
#7 0x00160a80 in gc_collect_with_callback (
tstate=0x5796d0 <_PyRuntime+235648>, generation=2)
at ../Modules/gcmodule.c:1426
#8 0x001924f4 in PyGC_Collect () at ../Modules/gcmodule.c:2111
#9 0x0017b93c in Py_FinalizeEx () at ../Python/pylifecycle.c:1885
#10 0x0018a3ce in Py_RunMain () at ../Modules/main.c:711
#11 0x0014ea00 in Py_BytesMain (argc=6, argv=<optimized out>)
at ../Modules/main.c:763
#12 0xb6e517da in ?? () from /lib/arm-linux-gnueabihf/libc.so.6
#13 0xb6e5187e in __libc_start_main () from /lib/arm-linux-gnueabihf/libc.so.6
#14 0x0014e8e4 in _start ()
Backtrace stopped: previous frame identical to this frame (corrupt stack?)It seems fine with Python 3.11, though, so there is some change in Python 3.12 which is causing this corruption. But what I don't know is whether it is a bytecode issue or, as is seeming more likely now, a Python 3.12 issue.
P403n1x87
commented
11 months ago Backtrace stopped: previous frame identical to this frame (corrupt stack?)
This seems to be another cue for frame stack corruption. I've had these in the past with bad bytecode from my side, so that's why I think this could be due to some inaccurate bytecode reconstruction in at least one of the tests. What I find strange is how this is not being caught by the framework test 🤔
juliangilbey
commented
11 months ago Backtrace stopped: previous frame identical to this frame (corrupt stack?)
This seems to be another cue for frame stack corruption. I've had these in the past with bad bytecode from my side, so that's why I think this could be due to some inaccurate bytecode reconstruction in at least one of the tests. What I find strange is how this is not being caught by the framework test 🤔
It's also weird that of all the architectures it's being tested on (I'm not sure exactly which yet), I've only received a report that it fails on armhf. It certainly succeeds on amd64. If you'd like me to test it on other architectures, I could.
MatthieuDartiailh
commented
11 months ago I am honestly not sure what is going on here. To make sure bytecode is really in cause, could you pick one offending test and disable any recompiled bytecode execution and see if the error persist ? You can use pytest-repeat to easily run multiple times the same test. Anotrher thing you could attempt is to create an auto-use fixture running the GC manually to see if it makes the failure more consistent.
juliangilbey
commented
11 months ago Hi @MatthieuDartiailh, thanks for taking a look at this!
I'm not sure I fully understand your request, I'm afraid, but I'll give it my best shot. Here is what I've tried; the results are weird, but perhaps this is informative? To ensure I've not missed anything, I've patched up to the most recent commit: f586bae
I was consistently finding that it crashed, reporting that it was in test_cfg.py::test_huge_code_with_numerous_blocks. So I modified the final lines of the test:
bytecode = Bytecode(mk_if_then_else(5000))
bytecode.compute_stacksize() in three ways:
bytecode.compute_stacksize() call. The test still crashed at the same place.Bytecode call by instructs = mk_if_then_else(5000) (and still commented out the final line). The test still crashed at the same place.pass. The test now passed, and the crash occurred at test_roundtrip_exception_handling.Next, I tried creating an autouse fixture:
# contents of tests/conftest.py
import gc
import pytest
@pytest.fixture(autouse=True)
def run_gc():
yield
gc.collect()and now running pytest with Python 3.12 consistently crashes at tests/test_bytecode.py::test_negative_size_binary_with_disable_check_of_pre_and_post. So I commented out the final two lines of that test (the lines co = code.to_code(...) and self.assertEqual(...)) and now everything runs without trouble.
I then removed conftest.py, leaving the test_negative_size_binary_with_disable_check_of_pre_and_post test disabled, and the pytest run still runs smoothly. So it looks as though this may be the source of the issue, though I have nowhere near enough understanding to know why.
Finally, I left the test unpatched and ran pytest with the option --deselect=tests/test_bytecode.py::test_negative_size_binary_with_disable_check_of_pre_and_post. But then it crashed at test_huge_code_with_numerous_blocks once again, which is just weird.
This is a very strange one, and might not be the fault of bytecode:
It seems to run fine on other architectures, so I don't know what's up here.