search

pybind / pybind11

Seamless operability between C++11 and Python
https://pybind11.readthedocs.io/
Other
15.39k stars 2.08k forks source link

[BUG] 🐍 3.10-dev (ubuntu, mac, ~windows): TypeError: 'str' object is not callable #2774

Open EricCousineau-TRI opened 3 years ago

EricCousineau-TRI commented 3 years ago

Seen this crop up in my PR and another:

/home/runner/work/pybind11/pybind11/tests/test_embed/test_interpreter.cpp:244: FAILED:
  {Unknown expression after the reported line}
due to unexpected exception with message:
  TypeError: 'str' object is not callable

Example PRs:

Guessing it's just a minor thing that'll blow over. I guess we can close this if we don't see it reoccur?

FYI @henryiii @YannickJadoul

YannickJadoul commented 3 years ago

Already spotted this in #2768 as well, but haven't had time to look closer. We might not want to wait until 3.10 is released to report this (ahem we'd never let such a thing happen cough) ;-)

henryiii commented 3 years ago

Before looking at the code, this looked like something is being passed through as a type annotation and being called, but now type annotations are all strings, so it's now calling a string instead of a callable. But have no clue where anything like that would be in our code. It seems to possibly be happening when a module is being reloaded? That could be an issue somewhere in Python's stdlib code.

I think 3.10a4 is the first with "from __future__ import annotations" on by default, which is also suspicious.

henryiii commented 3 years ago

We don't run these tests on Windows after 3.7 due to the DLL loading changes which we've never worked around, IIRC. So it's likely a problem everywhere.

EricCousineau-TRI commented 3 years ago

We don't run these tests on Windows after 3.7 due to the DLL loading changes which we've never worked around, IIRC. So it's likely a problem everywhere.

Er, I understood your prior post, but I'm not sure how Windows 3.7 w/ DLL relates to 3.10-dev - can you expand on that?

EricCousineau-TRI commented 3 years ago

Oh - meaning I should remove the (ubuntu, mac) specifier. Derp.

bstaletic commented 3 years ago

We don't run these tests on Windows after 3.7 due to the DLL loading changes which we've never worked around, IIRC. So it's likely a problem everywhere.

In 3.8+ loading DLLs on Winblows changes in this way (os.add_dll_directory):

https://docs.python.org/3/whatsnew/3.8.html#changes-in-the-python-api https://docs.python.org/3/whatsnew/3.8.html#os https://docs.python.org/3/whatsnew/3.8.html#ctypes https://docs.python.org/3/library/os.html?highlight=add_dll_directory#os.add_dll_directory

Here's add_dll_directory used in my project: https://github.com/ycm-core/ycmd/pull/1400/commits/6c28224b5ff5dca3cf1d72c02615c2da317ba4c6

@henryiii tried to fix this for pybind11, but with no success.

bstaletic commented 3 years ago

this looked like something is being passed through as a type annotation and being called, but now type annotations are all strings

That would mean this would be reproducible on python 3.9.1 with from __future__ import annotations. I tried adding that line to all tests/**/test_*.py files, but make cpptest was successful

YannickJadoul commented 3 years ago

I just tried, but I can't reproduce locally with the last version of Python. I installed "Python 3.10.0a4+ (heads/master:d16f617, Jan 9 2021, 20:40:43)" through pyenv, and all tests pass. I'll give the exact same version, 3.10.0a4, a try as well.

YannickJadoul commented 3 years ago

I just tried, but I can't reproduce locally with the last version of Python. I installed "Python 3.10.0a4+ (heads/master:d16f617, Jan 9 2021, 20:40:43)" through pyenv, and all tests pass. I'll give the exact same version, 3.10.0a4, a try as well.

Scratch that. It's the embedding tests that fail. If you run those, CPython's master still fails.

henryiii commented 3 years ago

I suspect it's loading a file somewhere in Python, not our .py's. We don't use annotations, sadly, much less try to call them.

henryiii commented 3 years ago

Is there some way to get more information about where the failure actually is, perhaps? If we just had a filename, that would be huge. We need from rich.traceback import install; install(show_locals=True), perhaps :)

YannickJadoul commented 3 years ago

Seems like something worse is going on, though, I'm afraid. If I comment out preceding tests in test_interpreter.cpp, I'm getting a beautiful SIGABRT:

Fatal Python error: compiler_make_closure: lookup '__class__' in DerivedWidget 5 -1
freevars of __init__: ('__class__',)
YannickJadoul commented 3 years ago
gdb backtrace for the interested (only just started investigating) ``` (gdb) r Starting program: /home/yannick/pybind11/build_310/tests/test_embed/test_embed [Thread debugging using libthread_db enabled] Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1". Fatal Python error: compiler_make_closure: lookup '__class__' in DerivedWidget 5 -1 freevars of __init__: ('__class__',) Python runtime state: initialized Current thread 0x00007ffff7fc1740 (most recent call first): File "", line 241 in _call_with_frames_removed File "", line 897 in source_to_code File "", line 967 in get_code File "", line 829 in exec_module File "", line 698 in _load_unlocked File "", line 1005 in _find_and_load_unlocked File "", line 1026 in _find_and_load Program received signal SIGABRT, Aborted. __GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:51 51 ../sysdeps/unix/sysv/linux/raise.c: No such file or directory. (gdb) bt #0 __GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:51 #1 0x00007ffff664b921 in __GI_abort () at abort.c:79 #2 0x00007ffff77f3640 in fatal_error_exit (status=-1) at ../Python/pylifecycle.c:2488 #3 fatal_error (stream=stream@entry=0x7ffff69f7680 <_IO_2_1_stderr_>, header=header@entry=0, prefix=prefix@entry=0x0, msg=msg@entry=0x0, status=status@entry=-1) at ../Python/pylifecycle.c:2572 #4 0x00007ffff77f69a1 in _Py_FatalErrorFormat (func=func@entry=0x7ffff798ba00 <__func__.17465> "compiler_make_closure", format=format@entry=0x7ffff798c878 "lookup %s in %s %d %d\nfreevars of %s: %s\n") at ../Python/pylifecycle.c:2622 #5 0x00007ffff77bd99b in compiler_make_closure (c=c@entry=0x7fffffffb800, co=co@entry=0x7ffff7e68110, flags=flags@entry=0, qualname=qualname@entry='DerivedWidget__doc____init__') at ../Python/compile.c:1915 #6 0x00007ffff77c459f in compiler_function (c=0x7fffffffb800, s=, is_async=is_async@entry=0) at ../Python/compile.c:2260 #7 0x00007ffff77c4766 in compiler_visit_stmt (c=c@entry=0x7fffffffb800, s=0x55555590df40) at ../Python/compile.c:3332 #8 0x00007ffff77c4a0c in compiler_body (c=c@entry=0x7fffffffb800, stmts=0x55555590f828) at ../Python/compile.c:1803 #9 0x00007ffff77c4de8 in compiler_class (c=0x7fffffffb800, s=0x55555590f868) at ../Python/compile.c:2335 #10 0x00007ffff77c4776 in compiler_visit_stmt (c=c@entry=0x7fffffffb800, s=0x55555590f868) at ../Python/compile.c:3334 #11 0x00007ffff77c4a0c in compiler_body (c=c@entry=0x7fffffffb800, stmts=0x55555590fb48) at ../Python/compile.c:1803 #12 0x00007ffff77c5173 in compiler_mod (c=c@entry=0x7fffffffb800, mod=mod@entry=0x55555590fb68) at ../Python/compile.c:1823 #13 0x00007ffff77c7063 in PyAST_CompileObject (mod=0x55555590fb68, filename=filename@entry='./test_interpreter.py', flags=flags@entry=0x7fffffffb8d8, optimize=optimize@entry=-1, arena=arena@entry=0x7ffff59e8fa0) at ../Python/compile.c:392 #14 0x00007ffff77fd7b0 in Py_CompileStringObject ( str=0x7ffff7eb5cc0 "# -*- coding: utf-8 -*-\nfrom widget_module import Widget\n\n\nclass DerivedWidget(Widget):\n def __init__(self, message):\n super(DerivedWidget, self).__init__(message)\n\n def the_answer(self):"..., filename=filename@entry='./test_interpreter.py', start=257, flags=flags@entry=0x7fffffffb8d8, optimize=optimize@entry=-1) at ../Python/pythonrun.c:1307 #15 0x00007ffff779c157 in builtin_compile_impl (module=module@entry=, source=source@entry=b'# -*- coding: utf-8 -*-\nfrom widget_module import Widget\n\n\nclass DerivedWidget(Widget):\n def __init__(self, message):\n super(DerivedWidget, self).__init__(message)\n\n def the_answer(self):\n return 42\n', filename='./test_interpreter.py', mode=, mode@entry=0x7ffff5a78f20 "exec", flags=flags@entry=0, dont_inherit=dont_inherit@entry=1, optimize=-1, feature_version=-1) at ../Python/bltinmodule.c:819 #16 0x00007ffff779c4a7 in builtin_compile (module=, args=, nargs=, kwnames=) at ../Python/clinic/bltinmodule.c.h:249 #17 0x00007ffff76f10ad in cfunction_vectorcall_FASTCALL_KEYWORDS (func=, args=0x7ffff7f15228, nargsf=, kwnames=('dont_inherit', 'optimize')) at ../Objects/methodobject.c:442 #18 0x00007ffff76999be in PyVectorcall_Call (callable=callable@entry=, tuple=tuple@entry=(b'# -*- coding: utf-8 -*-\nfrom widget_module import Widget\n\n\nclass DerivedWidget(Widget):\n def __init__(self, message):\n super(DerivedWidget, self).__init__(message)\n\n def the_answer(self):\n return 42\n', './test_interpreter.py', 'exec'), kwargs=kwargs@entry={'dont_inherit': True, 'optimize': -1}) at ../Objects/call.c:243 #19 0x00007ffff7699c06 in _PyObject_Call (tstate=0x5555558154b0, callable=callable@entry=, args=args@entry=(b'# -*- coding: utf-8 -*-\nfrom widget_module import Widget\n\n\nclass DerivedWidget(Widget):\n def __init__(self, message):\n super(DerivedWidget, self).__init__(message)\n\n def the_answer(self):\n return 42\n', './test_interpreter.py', 'exec'), kwargs=kwargs@entry={'dont_inherit': True, 'optimize': -1}) at ../Objects/call.c:266 #20 0x00007ffff7699c83 in PyObject_Call (callable=callable@entry=, args=args@entry=(b'# -*- coding: utf-8 -*-\nfrom widget_module import Widget\n\n\nclass DerivedWidget(Widget):\n def __init__(self, message):\n super(DerivedWidget, self).__init__(message)\n\n def the_answer(self):\n return 42\n', './test_interpreter.py', 'exec'), kwargs=kwargs@entry={'dont_inherit': True, 'optimize': -1}) at ../Objects/call.c:293 #21 0x00007ffff77a5878 in do_call_core (tstate=tstate@entry=0x5555558154b0, func=func@entry=, callargs=callargs@entry=(b'# -*- coding: utf-8 -*-\nfrom widget_module import Widget\n\n\nclass DerivedWidget(Widget):\n def __init__(self, message):\n super(DerivedWidget, self).__init__(message)\n\n def the_answer(self):\n return 42\n', './test_interpreter.py', 'exec'), kwdict=kwdict@entry={'dont_inherit': True, 'optimize': -1}) at ../Python/ceval.c:5381 #22 0x00007ffff77b0e08 in _PyEval_EvalFrameDefault (tstate=0x5555558154b0, f=Frame 0x7ffff7eb63f0, for file , line 241, in _call_with_frames_removed (f=, args=(b'# -*- coding: utf-8 -*-\nfrom widget_module import Widget\n\n\nclass DerivedWidget(Widget):\n def __init__(self, message):\n super(DerivedWidget, self).__init__(message)\n\n def the_answer(self):\n return 42\n', './test_interpreter.py', 'exec'), kwds={'dont_inherit': True, 'optimize': -1}), throwflag=) at ../Python/ceval.c:3860 #23 0x00007ffff77b2e7e in _PyEval_EvalFrame (throwflag=0, f=Frame 0x7ffff7eb63f0, for file , line 241, in _call_with_frames_removed (f=, args=(b'# -*- coding: utf-8 -*-\nfrom widget_module import Widget\n\n\nclass DerivedWidget(Widget):\n def __init__(self, message):\n super(DerivedWidget, self).__init__(message)\n\n def the_answer(self):\n return 42\n', './test_interpreter.py', 'exec'), kwds={'dont_inherit': True, 'optimize': -1}), tstate=0x5555558154b0) at ../Include/internal/pycore_ceval.h:40 #24 _PyEval_EvalCode (tstate=0x5555558154b0, _co=, globals=, locals=locals@entry=0x0, args=args@entry=0x7ffff5a5c3c8, argcount=4, kwnames=0x7ffff7ebad38, kwargs=0x7ffff5a5c3e8, kwcount=2, kwstep=1, defs=0x0, defcount=0, kwdefs=0x0, closure=0x0, name='_call_with_frames_removed', qualname='_call_with_frames_removed') at ../Python/ceval.c:4620 #25 0x00007ffff7699de2 in _PyFunction_Vectorcall (func=, stack=0x7ffff5a5c3c8, nargsf=, kwnames=) at ../Objects/call.c:396 #26 0x00007ffff77a557a in _PyObject_VectorcallTstate (kwnames=('dont_inherit', 'optimize'), nargsf=9223372036854775812, args=0x7ffff5a5c3c8, callable=, tstate=0x5555558154b0) at ../Include/cpython/abstract.h:114 #27 PyObject_Vectorcall (kwnames=('dont_inherit', 'optimize'), nargsf=9223372036854775812, args=0x7ffff5a5c3c8, callable=) at ../Include/cpython/abstract.h:123 #28 call_function (tstate=tstate@entry=0x5555558154b0, pp_stack=pp_stack@entry=0x7fffffffbe68, oparg=oparg@entry=6, kwnames=kwnames@entry=('dont_inherit', 'optimize')) at ../Python/ceval.c:5361 #29 0x00007ffff77b0b57 in _PyEval_EvalFrameDefault (tstate=0x5555558154b0, --Type for more, q to quit, c to continue without paging-- f=Frame 0x7ffff5a5c240, for file , line 897, in source_to_code (self=, data=b'# -*- coding: utf-8 -*-\nfrom widget_module import Widget\n\n\nclass DerivedWidget(Widget):\n def __init__(self, message):\n super(DerivedWidget, self).__init__(message)\n\n def the_answer(self):\n return 42\n', path='./test_interpreter.py', _optimize=-1), throwflag=) at ../Python/ceval.c:3815 #30 0x00007ffff77b2e7e in _PyEval_EvalFrame (throwflag=0, f=Frame 0x7ffff5a5c240, for file , line 897, in source_to_code (self=, data=b'# -*- coding: utf-8 -*-\nfrom widget_module import Widget\n\n\nclass DerivedWidget(Widget):\n def __init__(self, message):\n super(DerivedWidget, self).__init__(message)\n\n def the_answer(self):\n return 42\n', path='./test_interpreter.py', _optimize=-1), tstate=0x5555558154b0) at ../Include/internal/pycore_ceval.h:40 #31 _PyEval_EvalCode (tstate=0x5555558154b0, _co=, globals=, locals=locals@entry=0x0, args=args@entry=0x555555884ad0, argcount=3, kwnames=0x0, kwargs=0x555555884ae8, kwcount=0, kwstep=1, defs=0x0, defcount=0, kwdefs={'_optimize': -1}, closure=0x0, name='source_to_code', qualname='SourceLoader.source_to_code') at ../Python/ceval.c:4620 #32 0x00007ffff7699de2 in _PyFunction_Vectorcall (func=, stack=0x555555884ad0, nargsf=, kwnames=) at ../Objects/call.c:396 #33 0x00007ffff77a557a in _PyObject_VectorcallTstate (kwnames=0x0, nargsf=9223372036854775811, args=0x555555884ad0, callable=, tstate=0x5555558154b0) at ../Include/cpython/abstract.h:114 #34 PyObject_Vectorcall (kwnames=0x0, nargsf=9223372036854775811, args=0x555555884ad0, callable=) at ../Include/cpython/abstract.h:123 #35 call_function (tstate=tstate@entry=0x5555558154b0, pp_stack=pp_stack@entry=0x7fffffffc138, oparg=oparg@entry=3, kwnames=kwnames@entry=0x0) at ../Python/ceval.c:5361 #36 0x00007ffff77b08ef in _PyEval_EvalFrameDefault (tstate=0x5555558154b0, f=Frame 0x5555558848f0, for file , line 967, in get_code (self=, fullname='test_interpreter', source_path='./test_interpreter.py', source_mtime=1602280047, source_bytes=b'# -*- coding: utf-8 -*-\nfrom widget_module import Widget\n\n\nclass DerivedWidget(Widget):\n def __init__(self, message):\n super(DerivedWidget, self).__init__(message)\n\n def the_answer(self):\n return 42\n', source_hash=None, hash_based=False, check_source=True, bytecode_path='./__pycache__/test_interpreter.cpython-310.pyc', st={'mtime': , 'size': 219}), throwflag=) at ../Python/ceval.c:3784 #37 0x00007ffff769958b in _PyEval_EvalFrame (throwflag=0, f=Frame 0x5555558848f0, for file , line 967, in get_code (self=, fullname='test_interpreter', source_path='./test_interpreter.py', source_mtime=1602280047, source_bytes=b'# -*- coding: utf-8 -*-\nfrom widget_module import Widget\n\n\nclass DerivedWidget(Widget):\n def __init__(self, message):\n super(DerivedWidget, self).__init__(message)\n\n def the_answer(self):\n return 42\n', source_hash=None, hash_based=False, check_source=True, bytecode_path='./__pycache__/test_interpreter.cpython-310.pyc', st={'mtime': , 'size': 219}), tstate=0x5555558154b0) at ../Include/internal/pycore_ceval.h:40 #38 function_code_fastcall (tstate=0x5555558154b0, co=, args=0x7ffff7ef61e0, args@entry=0x7ffff7ef61d0, nargs=2, globals=) at ../Objects/call.c:330 #39 0x00007ffff7699ee0 in _PyFunction_Vectorcall (func=, stack=0x7ffff7ef61d0, nargsf=, kwnames=) at ../Objects/call.c:367 #40 0x00007ffff77a557a in _PyObject_VectorcallTstate (kwnames=0x0, nargsf=9223372036854775810, args=0x7ffff7ef61d0, callable=, tstate=0x5555558154b0) at ../Include/cpython/abstract.h:114 #41 PyObject_Vectorcall (kwnames=0x0, nargsf=9223372036854775810, args=0x7ffff7ef61d0, callable=) at ../Include/cpython/abstract.h:123 #42 call_function (tstate=tstate@entry=0x5555558154b0, pp_stack=pp_stack@entry=0x7fffffffc368, oparg=oparg@entry=2, kwnames=kwnames@entry=0x0) at ../Python/ceval.c:5361 #43 0x00007ffff77b08ef in _PyEval_EvalFrameDefault (tstate=0x5555558154b0, f=Frame 0x7ffff7ef6050, for file , line 829, in exec_module (self=, module=), throwflag=) at ../Python/ceval.c:3784 #44 0x00007ffff769958b in _PyEval_EvalFrame (throwflag=0, f=Frame 0x7ffff7ef6050, for file , line 829, in exec_module (self=, module=), tstate=0x5555558154b0) at ../Include/internal/pycore_ceval.h:40 #45 function_code_fastcall (tstate=0x5555558154b0, co=, args=0x7ffff7e5bbe0, args@entry=0x7ffff7e5bbd0, nargs=2, globals=) at ../Objects/call.c:330 #46 0x00007ffff7699ee0 in _PyFunction_Vectorcall (func=, stack=0x7ffff7e5bbd0, nargsf=, kwnames=) at ../Objects/call.c:367 #47 0x00007ffff77a557a in _PyObject_VectorcallTstate (kwnames=0x0, nargsf=9223372036854775810, args=0x7ffff7e5bbd0, callable=, tstate=0x5555558154b0) at ../Include/cpython/abstract.h:114 #48 PyObject_Vectorcall (kwnames=0x0, nargsf=9223372036854775810, args=0x7ffff7e5bbd0, callable=) at ../Include/cpython/abstract.h:123 #49 call_function (tstate=tstate@entry=0x5555558154b0, pp_stack=pp_stack@entry=0x7fffffffc598, oparg=oparg@entry=2, kwnames=kwnames@entry=0x0) at ../Python/ceval.c:5361 #50 0x00007ffff77b08ef in _PyEval_EvalFrameDefault (tstate=0x5555558154b0, f=Frame 0x7ffff7e5ba50, for file , line 698, in _load_unlocked (spec=, origin='./test_interpreter.py', loader_state=None, submodule_search_locations=None, _set_fileattr=True, _cached='./__pycache__/test_interpreter.cpython-310.pyc', _initializing=True) at remote 0x7ffff7e8d2d0>, module=), throwflag=) at ../Python/ceval.c:3784 #51 0x00007ffff769958b in _PyEval_EvalFrame (throwflag=0, f=Frame 0x7ffff7e5ba50, for file , line 698, in _load_unlocked (spec=, origin='./test_interpreter.py', loader_state=None, submodule_search_locations=None, _set_fileattr=True, _cached='./__pycache__/test_interpreter.cpython-310.pyc', _initializing=True) at remote 0x7ffff7e8d2d0>, module=), tstate=0x5555558154b0) at ../Include/internal/pycore_ceval.h:40 #52 function_code_fastcall (tstate=0x5555558154b0, co=, args=0x5555558b9778, args@entry=0x5555558b9770, nargs=1, globals=) at ../Objects/call.c:330 #53 0x00007ffff7699ee0 in _PyFunction_Vectorcall (func=, stack=0x5555558b9770, nargsf=, kwnames=) at ../Objects/call.c:367 #54 0x00007ffff77a557a in _PyObject_VectorcallTstate (kwnames=0x0, nargsf=9223372036854775809, args=0x5555558b9770, callable=, tstate=0x5555558154b0) at ../Include/cpython/abstract.h:114 #55 PyObject_Vectorcall (kwnames=0x0, nargsf=9223372036854775809, args=0x5555558b9770, callable=) at ../Include/cpython/abstract.h:123 --Type for more, q to quit, c to continue without paging-- #56 call_function (tstate=tstate@entry=0x5555558154b0, pp_stack=pp_stack@entry=0x7fffffffc7d0, oparg=oparg@entry=1, kwnames=kwnames@entry=0x0) at ../Python/ceval.c:5361 #57 0x00007ffff77b0a30 in _PyEval_EvalFrameDefault (tstate=0x5555558154b0, f=Frame 0x5555558b95c0, for file , line 1005, in _find_and_load_unlocked (name='test_interpreter', import_=, path=None, parent='', spec=, origin='./test_interpreter.py', loader_state=None, submodule_search_locations=None, _set_fileattr=True, _cached='./__pycache__/test_interpreter.cpython-310.pyc', _initializing=True) at remote 0x7ffff7e8d2d0>), throwflag=) at ../Python/ceval.c:3798 #58 0x00007ffff769958b in _PyEval_EvalFrame (throwflag=0, f=Frame 0x5555558b95c0, for file , line 1005, in _find_and_load_unlocked (name='test_interpreter', import_=, path=None, parent='', spec=, origin='./test_interpreter.py', loader_state=None, submodule_search_locations=None, _set_fileattr=True, _cached='./__pycache__/test_interpreter.cpython-310.pyc', _initializing=True) at remote 0x7ffff7e8d2d0>), tstate=0x5555558154b0) at ../Include/internal/pycore_ceval.h:40 #59 function_code_fastcall (tstate=0x5555558154b0, co=, args=0x55555582e190, args@entry=0x55555582e180, nargs=2, globals=) at ../Objects/call.c:330 #60 0x00007ffff7699ee0 in _PyFunction_Vectorcall (func=, stack=0x55555582e180, nargsf=, kwnames=) at ../Objects/call.c:367 #61 0x00007ffff77a557a in _PyObject_VectorcallTstate (kwnames=0x0, nargsf=9223372036854775810, args=0x55555582e180, callable=, tstate=0x5555558154b0) at ../Include/cpython/abstract.h:114 #62 PyObject_Vectorcall (kwnames=0x0, nargsf=9223372036854775810, args=0x55555582e180, callable=) at ../Include/cpython/abstract.h:123 #63 call_function (tstate=tstate@entry=0x5555558154b0, pp_stack=pp_stack@entry=0x7fffffffca00, oparg=oparg@entry=2, kwnames=kwnames@entry=0x0) at ../Python/ceval.c:5361 #64 0x00007ffff77b0a30 in _PyEval_EvalFrameDefault (tstate=0x5555558154b0, f=Frame 0x55555582dff0, for file , line 1026, in _find_and_load (name='test_interpreter', import_=, module=), throwflag=) at ../Python/ceval.c:3798 #65 0x00007ffff769958b in _PyEval_EvalFrame (throwflag=0, f=Frame 0x55555582dff0, for file , line 1026, in _find_and_load (name='test_interpreter', import_=, module=), tstate=0x5555558154b0) at ../Include/internal/pycore_ceval.h:40 #66 function_code_fastcall (tstate=0x5555558154b0, co=, args=0x7fffffffcb60, args@entry=0x7fffffffcb50, nargs=2, globals=) at ../Objects/call.c:330 #67 0x00007ffff7699ee0 in _PyFunction_Vectorcall (func=, stack=0x7fffffffcb50, nargsf=, kwnames=) at ../Objects/call.c:367 #68 0x00007ffff769ab66 in _PyObject_VectorcallTstate (kwnames=0x0, nargsf=2, args=0x7fffffffcb50, callable=, tstate=0x5555558154b0) at ../Include/cpython/abstract.h:114 #69 object_vacall (tstate=tstate@entry=0x5555558154b0, base=base@entry=0x0, callable=, vargs=vargs@entry=0x7fffffffcbc8) at ../Objects/call.c:792 #70 0x00007ffff769aea6 in _PyObject_CallMethodIdObjArgs (obj=0x0, name=name@entry=0x7ffff7da8780 ) at ../Objects/call.c:883 #71 0x00007ffff77d8733 in import_find_and_load (tstate=tstate@entry=0x5555558154b0, abs_name=abs_name@entry='test_interpreter') at ../Python/import.c:1505 #72 0x00007ffff77dc463 in PyImport_ImportModuleLevelObject (name='test_interpreter', globals=, locals=, fromlist=[], level=0) at ../Python/import.c:1606 #73 0x00007ffff779a3fb in builtin___import__ (self=, args=, kwds=) at ../Python/bltinmodule.c:280 #74 0x00007ffff76f0f0e in cfunction_call (func=, args=, kwargs=) at ../Objects/methodobject.c:539 #75 0x00007ffff769a30d in _PyObject_MakeTpCall (tstate=tstate@entry=0x5555558154b0, callable=callable@entry=, args=args@entry=0x7fffffffce20, nargs=nargs@entry=5, keywords=keywords@entry=0x0) at ../Objects/call.c:191 #76 0x00007ffff769b687 in _PyObject_VectorcallTstate (kwnames=0x0, nargsf=5, args=0x7fffffffce20, callable=, tstate=0x5555558154b0) at ../Include/cpython/abstract.h:112 #77 _PyObject_CallFunctionVa (tstate=0x5555558154b0, callable=callable@entry=, format=format@entry=0x7ffff7990094 "OOOOi", va=va@entry=0x7fffffffce98, is_size_t=is_size_t@entry=0) at ../Objects/call.c:543 #78 0x00007ffff769b7a9 in PyObject_CallFunction (callable=callable@entry=, format=format@entry=0x7ffff7990094 "OOOOi") at ../Objects/call.c:565 #79 0x00007ffff77dcb9f in PyImport_Import (module_name=module_name@entry='test_interpreter') at ../Python/import.c:1807 #80 0x00007ffff77dcd74 in PyImport_ImportModule (name=) at ../Python/import.c:1218 #81 0x00005555555ac565 in pybind11::module_::import(char const*) () #82 0x00005555555a9bc0 in ____C_A_T_C_H____T_E_S_T____0() () #83 0x000055555556c9b5 in Catch::RunContext::invokeActiveTestCase() () #84 0x000055555557d686 in Catch::RunContext::runCurrentTest(std::__cxx11::basic_string, std::allocator >&, std::__cxx11::basic_string, std::allocator >&) () #85 0x00005555555822d6 in Catch::RunContext::runTest(Catch::TestCase const&) () #86 0x0000555555584aed in Catch::Session::runInternal() () #87 0x0000555555584cf8 in Catch::Session::run() () #88 0x00005555555682e2 in main () ```

Follow-up: Oh joy. Adding these two lines to the start of TEST_CASE("Restart the interpreter") { makes a difference:

py::finalize_interpreter();
py::initialize_interpreter();
YannickJadoul commented 3 years ago

Minimal reproducer:

TEST_CASE("Reproducer") {
    py::finalize_interpreter();
    py::initialize_interpreter();
    py::module_::import("external_module");
}

This looks like fun.

YannickJadoul commented 3 years ago

CPython-only reproducer:

#include <Python.h>

int main() {
    Py_InitializeEx(1);

    Py_Finalize();
    Py_InitializeEx(1);

    PyRun_SimpleString("class Widget: pass\n"
                       "class DerivedWidget(Widget):\n"
                       "    def __init__(self):\n"
                       "        super().__init__()\n");

    Py_Finalize();

    printf("Works\n");

    return 0;
}

If anyone's bored, mind trying this out?

YannickJadoul commented 3 years ago

Bisected Python to show the error probably got introduced here: python/cpython#20058. I guess I'll make a bug report; this doesn't seem like obvious code to debug...

YannickJadoul commented 3 years ago

Reported here: https://bugs.python.org/issue42882 Thanks @bstaletic for testing out!

YannickJadoul commented 3 years ago

Should already be fixed by python/cpython#24193 :-) Hopefully we can soon try this out.

YannickJadoul commented 3 years ago

Merged into CPython's master. https://www.python.org/dev/peps/pep-0619/ says a new alpha is expected around "2021-02-01". That's pretty long, though. How do we best "fix" CI, until then?

bstaletic commented 3 years ago

Isn't 3.10-dev the current 3.10 HEAD? We've never specified that we're running the latest alpha.

YannickJadoul commented 3 years ago

Isn't 3.10-dev the current 3.10 HEAD? We've never specified that we're running the latest alpha.

I don't think so. If I read things correctly on https://github.com/actions/setup-python, it's only using releases? (cfr. also https://github.com/actions/setup-python/blob/66319ca9fa6c19287a320c27bd980953e731b827/src/find-python.ts#L157)

henryiii commented 3 years ago

3.10-dev is the last pre-release, so 3.10.0a4. Should we disable this check till 3.10.0a5?

YannickJadoul commented 3 years ago

I'd be up for that, as long as we can somehow get a reminder that 3.10.0a5 is out?

YannickJadoul commented 3 years ago

Alternatively, temporarily switch to nightly builds, so we can catch things earlier, before alpha releases? Can pyenv install the latest master? But maybe that's not worth it.

On Linux, we can use deadsnakes, though. I think that one has nightly builds pre-built? See #2746.

henryiii commented 3 years ago

Can pyenv install the latest master?

Maybe, but it took a while to support 3.9 after the release, so I kind of doubt it. It's also slow.

deadsnakes

Yes, we could do that.

  • © Githubissues.
  • Githubissues is a development platform for aggregating issues.