search

wlav / cppyy

Other
387 stars 39 forks source link

SIGSEGV when completing a struct containing arrays #173

Closed fabbbbbbbb closed 8 months ago

fabbbbbbbb commented 1 year ago

with python-3.11 and the latest commit of cppyy, when pressing TAB to complete a struct instance, the interpreter crashes

>>> import cppyy
>>> cppyy.cppdef('''struct S {uint32_t a[4]; uint32_t b[4];};''')
True
>>> s = cppyy.gbl.S()
>>> s. *** Break *** segmentation violation

#0  0x00007fec8e6dcc3a in wait4 () from /lib/x86_64-linux-gnu/libc.so.6
#1  0x00007fec8e64bf67 in ?? () from /lib/x86_64-linux-gnu/libc.so.6
#2  0x00007fec8da1dc9a in CppyyLegacy::TUnixSystem::Exec (shellcmd=<optimized out>, this=0x5594d35cfa40) at /opt/cppyy-backend/cling/src/core/unix/src/TUnixSystem.cxx:1814
#3  CppyyLegacy::TUnixSystem::StackTrace (this=0x5594d35cfa40) at /opt/cppyy-backend/cling/src/core/unix/src/TUnixSystem.cxx:2104
#4  0x00007fec879dee73 in (anonymous namespace)::do_trace (sig=1) at src/clingwrapper.cxx:239
#5  (anonymous namespace)::TExceptionHandlerImp::HandleException (this=<optimized out>, sig=1) at src/clingwrapper.cxx:252
#6  0x00007fec8da1ca79 in CppyyLegacy::TUnixSystem::DispatchSignals (this=0x5594d35cfa40, sig=CppyyLegacy::kSigSegmentationViolation) at /opt/cppyy-backend/cling/src/core/unix/src/TUnixSystem.cxx:2744
#7  <signal handler called>
#8  0x00007fec852cc0a6 in CPyCppyy::CPPInstance::CreateExtension (this=<optimized out>) at src/CPPInstance.cxx:89
#9  CPyCppyy::CPPInstance::CreateExtension (this=this
entry=0x0) at src/CPPInstance.cxx:86
#10 CPyCppyy::CPPInstance::GetDatamemberCache (this=this
entry=0x0) at src/CPPInstance.cxx:194
#11 0x00007fec852c7319 in CPyCppyy::dm_get (dm=dm
entry=0x7fec84c3ef70, pyobj=pyobj
entry=0x0) at src/CPPDataMember.cxx:38
#12 0x00005594d23219ac in type_getattro (type=0x5594d56fb440, name=0x5594d26dd0a0 <const_str_b>) at Objects/typeobject.c:3970
#13 0x00007fec852d9674 in CPyCppyy::meta_getattro (pyclass=0x5594d56fb440, pyname=0x5594d26dd0a0 <const_str_b>) at src/CPPScope.cxx:309
#14 0x00005594d22f540e in _PyObject_LookupAttr (v=0x5594d56fb440, name=<optimized out>, result=result
entry=0x7ffebcda5be0) at Objects/object.c:961
#15 0x00005594d23f2caf in builtin_getattr (self=<optimized out>, args=0x7fec8e96b178, nargs=3) at Python/bltinmodule.c:1127
#16 0x00005594d21fcc2a in _PyEval_EvalFrameDefault (tstate=<optimized out>, frame=<optimized out>, throwflag=<optimized out>) at ./Include/cpython/methodobject.h:52
#17 0x00005594d23fa841 in _PyEval_EvalFrame (throwflag=0, frame=0x7fec8e96b020, tstate=0x5594d2768a18 <_PyRuntime+166328>) at ./Include/internal/pycore_ceval.h:73
#18 _PyEval_Vector (tstate=0x5594d2768a18 <_PyRuntime+166328>, func=<optimized out>, locals=<optimized out>, args=<optimized out>, argcount=<optimized out>, kwnames=<optimized out>) at Python/ceval.c:6438
#19 0x00005594d227b9db in _PyObject_VectorcallTstate (kwnames=<optimized out>, nargsf=<optimized out>, args=<optimized out>, callable=<optimized out>, tstate=<optimized out>) at ./Include/internal/pycore_call.h:92
#20 method_vectorcall (method=method
entry=0x7fec8e0a1080, args=args
entry=0x7ffebcda5e40, nargsf=<optimized out>, kwnames=kwnames
entry=0x0) at Objects/classobject.c:89
#21 0x00005594d2277d10 in _PyObject_VectorcallTstate (kwnames=0x0, nargsf=<optimized out>, args=0x7ffebcda5e40, callable=0x7fec8e0a1080, tstate=0x5594d2768a18 <_PyRuntime+166328>) at ./Include/internal/pycore_call.h:92
#22 _PyObject_CallFunctionVa (is_size_t=<optimized out>, va=<optimized out>, format=<optimized out>, callable=0x7fec8e0a1080, tstate=0x5594d2768a18 <_PyRuntime+166328>) at Objects/call.c:536
#23 _PyObject_CallFunctionVa (tstate=0x5594d2768a18 <_PyRuntime+166328>, callable=0x7fec8e0a1080, format=<optimized out>, va=<optimized out>, is_size_t=<optimized out>) at Objects/call.c:495
#24 0x00005594d227935e in PyObject_CallFunction (callable=<optimized out>, format=format
entry=0x7fec8e07a011 "Ni") at Objects/call.c:558
#25 0x00007fec8e077449 in on_completion (text=0x5594d51b2c70 "s.", state=0) at /opt/Python-3.11.3/Modules/readline.c:1119
#26 0x00007fec8e0379c3 in rl_completion_matches () from /lib/x86_64-linux-gnu/libreadline.so.8
#27 0x00007fec8e077331 in flex_complete (text=0x5594d51b2c70 "s.", start=<optimized out>, end=<optimized out>) at /opt/Python-3.11.3/Modules/readline.c:1187
#28 0x00007fec8e037ae7 in ?? () from /lib/x86_64-linux-gnu/libreadline.so.8
#29 0x00007fec8e037cd2 in rl_complete_internal () from /lib/x86_64-linux-gnu/libreadline.so.8
#30 0x00007fec8e02d707 in _rl_dispatch_subseq () from /lib/x86_64-linux-gnu/libreadline.so.8
#31 0x00007fec8e02dcc5 in readline_internal_char () from /lib/x86_64-linux-gnu/libreadline.so.8
#32 0x00007fec8e048d75 in rl_callback_read_char () from /lib/x86_64-linux-gnu/libreadline.so.8
#33 0x00007fec8e076f55 in readline_until_enter_or_signal (signal=<synthetic pointer>, prompt=<optimized out>) at /opt/Python-3.11.3/Modules/readline.c:1352
#34 call_readline (sys_stdin=<optimized out>, sys_stdout=<optimized out>, prompt=<optimized out>) at /opt/Python-3.11.3/Modules/readline.c:1398
#35 0x00005594d22395a6 in PyOS_Readline (sys_stdin=0x7fec8e7e6980 <_IO_2_1_stdin_>, sys_stdout=0x7fec8e7e76a0 <_IO_2_1_stdout_>, prompt=0x7fec8e2d31a0 ">>> ") at Parser/myreadline.c:392
#36 0x00005594d223afb2 in tok_underflow_interactive (tok=0x5594d56d0570) at Parser/tokenizer.c:880
#37 tok_nextc (tok=0x5594d56d0570) at Parser/tokenizer.c:1064
#38 0x00005594d223c648 in tok_get (tok=tok
entry=0x5594d56d0570, p_start=p_start
entry=0x7ffebcda6f28, p_end=p_end
entry=0x7ffebcda6f30) at Parser/tokenizer.c:1419
#39 0x00005594d223e0dd in _PyTokenizer_Get (tok=0x5594d56d0570, p_start=p_start
entry=0x7ffebcda6f28, p_end=p_end
entry=0x7ffebcda6f30) at Parser/tokenizer.c:2121
#40 0x00005594d220249d in _PyPegen_fill_token (p=p
entry=0x7fec8ddc97f0) at Parser/pegen.c:213
#41 0x00005594d2235bf0 in statement_newline_rule (p=0x7fec8ddc97f0) at Parser/parser.c:1407
#42 interactive_rule (p=0x7fec8ddc97f0) at Parser/parser.c:1108
#43 _PyPegen_parse (p=p
entry=0x7fec8ddc97f0) at Parser/parser.c:38708
#44 0x00005594d22038f1 in _PyPegen_run_parser (p=0x7fec8ddc97f0) at Parser/pegen.c:837
#45 0x00005594d2203db7 in _PyPegen_run_parser_from_file_pointer (fp=fp
entry=0x7fec8e7e6980 <_IO_2_1_stdin_>, start_rule=start_rule
entry=256, filename_ob=filename_ob
entry=0x7fec8e2d3130, enc=enc
entry=0x7fec8e2d1be0 "utf-8", ps1=<optimized out>, ps1
entry=0x1e000000160 <error: Cannot access memory at address 0x1e000000160>, ps2=ps2
entry=0xe0000001a0 <error: Cannot access memory at address 0xe0000001a0>, flags=0x7ffebcda7188, errcode=0x7ffebcda70b4, arena=0x7fec8e1fb890) at Parser/pegen.c:910
#46 0x00005594d223922c in _PyParser_ASTFromFile (fp=fp
entry=0x7fec8e7e6980 <_IO_2_1_stdin_>, filename_ob=filename_ob
entry=0x7fec8e2d3130, enc=enc
entry=0x7fec8e2d1be0 "utf-8", mode=mode
entry=256, ps1=0x1e000000160 <error: Cannot access memory at address 0x1e000000160>, ps1
entry=0x7fec8e2d31a0 ">>> ", ps2=0xe0000001a0 <error: Cannot access memory at address 0xe0000001a0>, ps2
entry=0x7fec8e2d33a0 "... ", flags=<optimized out>, errcode=<optimized out>, arena=<optimized out>) at Parser/peg_api.c:26
#47 0x00005594d245d7c1 in PyRun_InteractiveOneObjectEx (fp=fp
entry=0x7fec8e7e6980 <_IO_2_1_stdin_>, filename=filename
entry=0x7fec8e2d3130, flags=flags
entry=0x7ffebcda7188) at Python/pythonrun.c:241
#48 0x00005594d245eba9 in _PyRun_InteractiveLoopObject (fp=fp
entry=0x7fec8e7e6980 <_IO_2_1_stdin_>, filename=filename
entry=0x7fec8e2d3130, flags=flags
entry=0x7ffebcda7188) at Python/pythonrun.c:138
#49 0x00005594d245f8c2 in _PyRun_AnyFileObject (flags=<optimized out>, closeit=<optimized out>, filename=0x7fec8e2d3130, fp=<optimized out>) at Python/pythonrun.c:73
#50 PyRun_AnyFileExFlags (fp=0x7fec8e7e6980 <_IO_2_1_stdin_>, filename=filename
entry=0x5594d25204b1 "<stdin>", closeit=closeit
entry=0, flags=flags
entry=0x7ffebcda7188) at Python/pythonrun.c:105
#51 0x00005594d248ae20 in pymain_run_stdin (config=0x5594d274ea60 <_PyRuntime+59904>) at Modules/main.c:509
#52 pymain_run_python (exitcode=0x7ffebcda72e0) at Modules/main.c:604
#53 0x00005594d248b25f in Py_RunMain () at Modules/main.c:680
#54 pymain_main (args=0x7ffebcda72a0) at Modules/main.c:710
#55 Py_BytesMain (argc=<optimized out>, argv=<optimized out>) at Modules/main.c:734
#56 0x00007fec8e61e083 in __libc_start_main () from /lib/x86_64-linux-gnu/libc.so.6
#57 0x00005594d2201cbe in _start ()
 *** Break *** segmentation violation
#0  0x00007fec8e6dcc3a in wait4 () from /lib/x86_64-linux-gnu/libc.so.6
#1  0x00007fec8e64bf67 in ?? () from /lib/x86_64-linux-gnu/libc.so.6
#2  0x00007fec8da1dc9a in CppyyLegacy::TUnixSystem::Exec (shellcmd=<optimized out>, this=0x5594d35cfa40) at /opt/cppyy-backend/cling/src/core/unix/src/TUnixSystem.cxx:1814
#3  CppyyLegacy::TUnixSystem::StackTrace (this=0x5594d35cfa40) at /opt/cppyy-backend/cling/src/core/unix/src/TUnixSystem.cxx:2104
#4  0x00007fec879decf3 in (anonymous namespace)::do_trace (sig=1) at src/clingwrapper.cxx:239
#5  (anonymous namespace)::TExceptionHandlerImp::HandleException (this=<optimized out>, sig=1) at src/clingwrapper.cxx:258
#6  0x00007fec8da1ca79 in CppyyLegacy::TUnixSystem::DispatchSignals (this=0x5594d35cfa40, sig=CppyyLegacy::kSigSegmentationViolation) at /opt/cppyy-backend/cling/src/core/unix/src/TUnixSystem.cxx:2744
#7  <signal handler called>
#8  0x00007fec852cc0a6 in CPyCppyy::CPPInstance::CreateExtension (this=<optimized out>) at src/CPPInstance.cxx:89
#9  CPyCppyy::CPPInstance::CreateExtension (this=this
entry=0x0) at src/CPPInstance.cxx:86
#10 CPyCppyy::CPPInstance::GetDatamemberCache (this=this
entry=0x0) at src/CPPInstance.cxx:194
#11 0x00007fec852c7319 in CPyCppyy::dm_get (dm=dm
entry=0x7fec84c3ef70, pyobj=pyobj
entry=0x0) at src/CPPDataMember.cxx:38
#12 0x00005594d23219ac in type_getattro (type=0x5594d56fb440, name=0x5594d26dd0a0 <const_str_b>) at Objects/typeobject.c:3970
#13 0x00007fec852d9674 in CPyCppyy::meta_getattro (pyclass=0x5594d56fb440, pyname=0x5594d26dd0a0 <const_str_b>) at src/CPPScope.cxx:309
#14 0x00005594d22f540e in _PyObject_LookupAttr (v=0x5594d56fb440, name=<optimized out>, result=result
entry=0x7ffebcda5be0) at Objects/object.c:961
#15 0x00005594d23f2caf in builtin_getattr (self=<optimized out>, args=0x7fec8e96b178, nargs=3) at Python/bltinmodule.c:1127
#16 0x00005594d21fcc2a in _PyEval_EvalFrameDefault (tstate=<optimized out>, frame=<optimized out>, throwflag=<optimized out>) at ./Include/cpython/methodobject.h:52
#17 0x00005594d23fa841 in _PyEval_EvalFrame (throwflag=0, frame=0x7fec8e96b020, tstate=0x5594d2768a18 <_PyRuntime+166328>) at ./Include/internal/pycore_ceval.h:73
#18 _PyEval_Vector (tstate=0x5594d2768a18 <_PyRuntime+166328>, func=<optimized out>, locals=<optimized out>, args=<optimized out>, argcount=<optimized out>, kwnames=<optimized out>) at Python/ceval.c:6438
#19 0x00005594d227b9db in _PyObject_VectorcallTstate (kwnames=<optimized out>, nargsf=<optimized out>, args=<optimized out>, callable=<optimized out>, tstate=<optimized out>) at ./Include/internal/pycore_call.h:92
#20 method_vectorcall (method=method
entry=0x7fec8e0a1080, args=args
entry=0x7ffebcda5e40, nargsf=<optimized out>, kwnames=kwnames
entry=0x0) at Objects/classobject.c:89
#21 0x00005594d2277d10 in _PyObject_VectorcallTstate (kwnames=0x0, nargsf=<optimized out>, args=0x7ffebcda5e40, callable=0x7fec8e0a1080, tstate=0x5594d2768a18 <_PyRuntime+166328>) at ./Include/internal/pycore_call.h:92
#22 _PyObject_CallFunctionVa (is_size_t=<optimized out>, va=<optimized out>, format=<optimized out>, callable=0x7fec8e0a1080, tstate=0x5594d2768a18 <_PyRuntime+166328>) at Objects/call.c:536
#23 _PyObject_CallFunctionVa (tstate=0x5594d2768a18 <_PyRuntime+166328>, callable=0x7fec8e0a1080, format=<optimized out>, va=<optimized out>, is_size_t=<optimized out>) at Objects/call.c:495
#24 0x00005594d227935e in PyObject_CallFunction (callable=<optimized out>, format=format
entry=0x7fec8e07a011 "Ni") at Objects/call.c:558
#25 0x00007fec8e077449 in on_completion (text=0x5594d51b2c70 "s.", state=0) at /opt/Python-3.11.3/Modules/readline.c:1119
#26 0x00007fec8e0379c3 in rl_completion_matches () from /lib/x86_64-linux-gnu/libreadline.so.8
#27 0x00007fec8e077331 in flex_complete (text=0x5594d51b2c70 "s.", start=<optimized out>, end=<optimized out>) at /opt/Python-3.11.3/Modules/readline.c:1187
#28 0x00007fec8e037ae7 in ?? () from /lib/x86_64-linux-gnu/libreadline.so.8
#29 0x00007fec8e037cd2 in rl_complete_internal () from /lib/x86_64-linux-gnu/libreadline.so.8
#30 0x00007fec8e02d707 in _rl_dispatch_subseq () from /lib/x86_64-linux-gnu/libreadline.so.8
#31 0x00007fec8e02dcc5 in readline_internal_char () from /lib/x86_64-linux-gnu/libreadline.so.8
#32 0x00007fec8e048d75 in rl_callback_read_char () from /lib/x86_64-linux-gnu/libreadline.so.8
#33 0x00007fec8e076f55 in readline_until_enter_or_signal (signal=<synthetic pointer>, prompt=<optimized out>) at /opt/Python-3.11.3/Modules/readline.c:1352
#34 call_readline (sys_stdin=<optimized out>, sys_stdout=<optimized out>, prompt=<optimized out>) at /opt/Python-3.11.3/Modules/readline.c:1398
#35 0x00005594d22395a6 in PyOS_Readline (sys_stdin=0x7fec8e7e6980 <_IO_2_1_stdin_>, sys_stdout=0x7fec8e7e76a0 <_IO_2_1_stdout_>, prompt=0x7fec8e2d31a0 ">>> ") at Parser/myreadline.c:392
#36 0x00005594d223afb2 in tok_underflow_interactive (tok=0x5594d56d0570) at Parser/tokenizer.c:880
#37 tok_nextc (tok=0x5594d56d0570) at Parser/tokenizer.c:1064
#38 0x00005594d223c648 in tok_get (tok=tok
entry=0x5594d56d0570, p_start=p_start
entry=0x7ffebcda6f28, p_end=p_end
entry=0x7ffebcda6f30) at Parser/tokenizer.c:1419
#39 0x00005594d223e0dd in _PyTokenizer_Get (tok=0x5594d56d0570, p_start=p_start
entry=0x7ffebcda6f28, p_end=p_end
entry=0x7ffebcda6f30) at Parser/tokenizer.c:2121
#40 0x00005594d220249d in _PyPegen_fill_token (p=p
entry=0x7fec8ddc97f0) at Parser/pegen.c:213
#41 0x00005594d2235bf0 in statement_newline_rule (p=0x7fec8ddc97f0) at Parser/parser.c:1407
#42 interactive_rule (p=0x7fec8ddc97f0) at Parser/parser.c:1108
#43 _PyPegen_parse (p=p
entry=0x7fec8ddc97f0) at Parser/parser.c:38708
#44 0x00005594d22038f1 in _PyPegen_run_parser (p=0x7fec8ddc97f0) at Parser/pegen.c:837
#45 0x00005594d2203db7 in _PyPegen_run_parser_from_file_pointer (fp=fp
entry=0x7fec8e7e6980 <_IO_2_1_stdin_>, start_rule=start_rule
entry=256, filename_ob=filename_ob
entry=0x7fec8e2d3130, enc=enc
entry=0x7fec8e2d1be0 "utf-8", ps1=<optimized out>, ps1
entry=0x1e000000160 <error: Cannot access memory at address 0x1e000000160>, ps2=ps2
entry=0xe0000001a0 <error: Cannot access memory at address 0xe0000001a0>, flags=0x7ffebcda7188, errcode=0x7ffebcda70b4, arena=0x7fec8e1fb890) at Parser/pegen.c:910
#46 0x00005594d223922c in _PyParser_ASTFromFile (fp=fp
entry=0x7fec8e7e6980 <_IO_2_1_stdin_>, filename_ob=filename_ob
entry=0x7fec8e2d3130, enc=enc
entry=0x7fec8e2d1be0 "utf-8", mode=mode
entry=256, ps1=0x1e000000160 <error: Cannot access memory at address 0x1e000000160>, ps1
entry=0x7fec8e2d31a0 ">>> ", ps2=0xe0000001a0 <error: Cannot access memory at address 0xe0000001a0>, ps2
entry=0x7fec8e2d33a0 "... ", flags=<optimized out>, errcode=<optimized out>, arena=<optimized out>) at Parser/peg_api.c:26
#47 0x00005594d245d7c1 in PyRun_InteractiveOneObjectEx (fp=fp
entry=0x7fec8e7e6980 <_IO_2_1_stdin_>, filename=filename
entry=0x7fec8e2d3130, flags=flags
entry=0x7ffebcda7188) at Python/pythonrun.c:241
#48 0x00005594d245eba9 in _PyRun_InteractiveLoopObject (fp=fp
entry=0x7fec8e7e6980 <_IO_2_1_stdin_>, filename=filename
entry=0x7fec8e2d3130, flags=flags
entry=0x7ffebcda7188) at Python/pythonrun.c:138
#49 0x00005594d245f8c2 in _PyRun_AnyFileObject (flags=<optimized out>, closeit=<optimized out>, filename=0x7fec8e2d3130, fp=<optimized out>) at Python/pythonrun.c:73
#50 PyRun_AnyFileExFlags (fp=0x7fec8e7e6980 <_IO_2_1_stdin_>, filename=filename
entry=0x5594d25204b1 "<stdin>", closeit=closeit
entry=0, flags=flags
entry=0x7ffebcda7188) at Python/pythonrun.c:105
#51 0x00005594d248ae20 in pymain_run_stdin (config=0x5594d274ea60 <_PyRuntime+59904>) at Modules/main.c:509
#52 pymain_run_python (exitcode=0x7ffebcda72e0) at Modules/main.c:604
#53 0x00005594d248b25f in Py_RunMain () at Modules/main.c:680
#54 pymain_main (args=0x7ffebcda72a0) at Modules/main.c:710
#55 Py_BytesMain (argc=<optimized out>, argv=<optimized out>) at Modules/main.c:734
#56 0x00007fec8e61e083 in __libc_start_main () from /lib/x86_64-linux-gnu/libc.so.6

Note that there is no such problem if the struct has simple members (ie, not arrays, like struct S {uint32_t a; uint32_t b;};). I don't remember having this issue with cppyy-2.3.

wlav commented 1 year ago

I can't reproduce it. Note that CreateExtension() should not have been called for simple tab-completion (it's not, as far as I can determine, when I run it), and the only way that CreateExtension() could crash, is if s is an invalid object, which doesn't seem likely in this simple scenario.

The CreateExtension() call, however, is related to the change, b/c it's called for memory views in the array case (uint32_t a[4]), but not for single elements (uint32_t a) or strings (as the resolved unsigned char was before). Memory views are expensive to create and often looped over, hence caching them is beneficial. Strings are copied, so no such problem. As such, I'm not excluding that something has changed for the worse, but I don't understand the flow.

wlav commented 9 months ago

I was able to reproduce it in a slightly different manner (at least, I think it's the same thing). Should be fixed now.

wlav commented 8 months ago

Should all be good now with release 3.1.0. Feel free to reopen if you find otherwise.