fabioz
commented
2 years ago It seems that Python 3.10.5 has some change which is related to this (I still haven't investigated, but I can see that it worked in 3.10.0 and is failing in 3.10.5).
Closed juliangilbey closed 1 year ago
fabioz
commented
2 years ago It seems that Python 3.10.5 has some change which is related to this (I still haven't investigated, but I can see that it worked in 3.10.0 and is failing in 3.10.5).
juliangilbey
commented
2 years ago Ouch, that sounds painful. Sorry for pointing this out; if I knew how to address it, I would certainly help :(
juliangilbey
commented
2 years ago I tried the latest version of PyDev.Debugger (commit 40a1c78a07c2a74e9a0da03433b3ad63d9c78c12) which includes the commit 1813bbd65cbe1382aae9d4851ad4bf14b55be1de which I understand was intended to fix this issue. Unfortunately it still doesn't work with Python 3.10.5:
(pydevd-venv) euler:~/debian/spyder-packages/pydevd/PyDev.Debugger-testing (main) $ PYDEVD_USE_CYTHON=YES pytest tests_python/test_bytecode_manipulation.py::test_set_pydevd_break_01
============================= test session starts ==============================
platform linux -- Python 3.10.5, pytest-7.1.2, pluggy-1.0.0+repack -- /usr/bin/python3
PYDEVD_USE_CYTHON: True
PYDEVD_TEST_VM: None
Number of processors: 32
Relevant system paths:
sys.executable: /usr/bin/python3
sys.prefix: /usr
sys.base_prefix: /usr
site.getusersitepackages(): /home/jdg/.local/lib/python3.10/site-packages
site.getsitepackages(): ['/usr/local/lib/python3.10/dist-packages', '/usr/lib/python3/dist-packages', '/usr/lib/python3.10/dist-packages']
Folder with "site-packages" in sys.path: /home/jdg/.local/lib/python3.10/site-packages
cachedir: .pytest_cache
PyQt5 5.15.7 -- Qt runtime 5.15.4 -- Qt compiled 5.15.4
hypothesis profile 'default' -> database=DirectoryBasedExampleDatabase('/home/jdg/debian/spyder-packages/pydevd/PyDev.Debugger-testing/.hypothesis/examples')
rootdir: /home/jdg/debian/spyder-packages/pydevd/PyDev.Debugger-testing, configfile: pytest.ini
plugins: cov-3.0.0, dependency-0.5.1, xdist-2.5.0, mock-3.8.2, forked-1.4.0, qt-4.0.2, flaky-3.7.0, anyio-3.6.1, order-1.0.1, timeout-2.1.0, asyncio-0.18.3, hypothesis-6.36.0, lazy-fixture-0.6.3, xvfb-2.0.0
asyncio: mode=legacy
collected 1 item
tests_python/test_bytecode_manipulation.py::test_set_pydevd_break_01 FAILED [100%]
=================================== FAILURES ===================================
___________________________ test_set_pydevd_break_01 ___________________________
def test_set_pydevd_break_01():
from tests_python.resources import _bytecode_overflow_example
> check('_bytecode_overflow_example.py', _bytecode_overflow_example.Dummy.fun, method_kwargs={'text': 'ing'}, has_line_event_optimized_in_original_case=True)
tests_python/test_bytecode_manipulation.py:185:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
filename = '_bytecode_overflow_example.py'
method = <function Dummy.fun at 0x7fb129ab71c0>, method_kwargs = {'text': 'ing'}
skip_breaks_at_lines = set()
method_to_change = <function Dummy.fun at 0x7fb129ab71c0>
stop_at_all_lines = False, has_line_event_optimized_in_original_case = True
def check(
filename,
method,
method_kwargs=None,
skip_breaks_at_lines=None,
method_to_change=None,
stop_at_all_lines=False,
has_line_event_optimized_in_original_case=False,
):
'''
:param has_line_event_optimized_in_original_case:
If True, we're handling a case where we have a double jump, i.e.: some case
where there's a JUMP_FORWARD which points to a JUMP_ABSOLUTE and this is
optimized so that the JUMP_FORWARD is changed directly to a JUMP_ABSOLUTE and
we end up skipping one line event which is supposed to be there but isn't in
the initial case but appears when we run after modifying the bytecode in memory.
See: https://github.com/microsoft/debugpy/issues/973#issuecomment-1178090731
'''
from _pydevd_frame_eval.pydevd_modify_bytecode import _get_code_line_info
from _pydevd_frame_eval import pydevd_modify_bytecode
if method_to_change is None:
method_to_change = method
if method_kwargs is None:
method_kwargs = {}
if skip_breaks_at_lines is None:
skip_breaks_at_lines = set()
pydev_break_stops = []
def _pydev_needs_stop_at_break(line):
pydev_break_stops.append(line)
return False
tracer = _Tracer()
def accept_frame(f):
return filename in f.f_code.co_filename
code = method_to_change.__code__
code_line_info = _get_code_line_info(code)
try:
tracer.accept_frame = accept_frame
def call():
method(**method_kwargs)
tracer.call(call)
breakpoint_hit_at_least_once = False
# Ok, we just ran the tracer once without any breakpoints.
#
# Gather its tracing profile: this will be our baseline for further tests (it should contain
# the events and the order in which the were executed).
#
# Note: methods cannot have random elements when executing (otherwise
# the order would be different and the test would be expected to fail).
baseline = tracer.stream.getvalue()
for line in sorted(code_line_info.line_to_offset):
if line in skip_breaks_at_lines:
continue
# Now, for each valid line, add a breakpoint and check if the tracing profile is exactly
# the same (and if the line where we added the breakpoint was executed, see if our
# callback got called).
success, new_code = pydevd_modify_bytecode.insert_pydevd_breaks(code, set([line]), _pydev_needs_stop_at_break=_pydev_needs_stop_at_break)
assert success
method_to_change.__code__ = new_code
tracer = _Tracer()
tracer.accept_frame = accept_frame
tracer.call(call)
contents = tracer.stream.getvalue()
assert tracer.lines_executed
if has_line_event_optimized_in_original_case:
lines = sorted(set(x[1] for x in dis.findlinestarts(new_code)))
new_line_contents = []
last_line = str(max(lines)) + ' '
for l in contents.splitlines(keepends=True):
if not l.strip().startswith(last_line):
new_line_contents.append(l)
contents = ''.join(new_line_contents)
if line in tracer.lines_executed:
assert set([line]) == set(pydev_break_stops)
breakpoint_hit_at_least_once = True
else:
if stop_at_all_lines:
raise AssertionError('Expected the debugger to stop at all lines. Did not stop at line: %s' % (line,))
del pydev_break_stops[:]
if baseline != contents:
print('------- replacement at line: %s ---------' % (line,))
print('------- baseline ---------')
print(baseline)
print('------- contents ---------')
print(contents)
print('-------- error -----------')
> assert baseline == contents
E AssertionError: assert ('18 fun _bytecode_overflow_example.py CALL \n'\n '20 fun _bytecode_overflow_example.py line \n'\n '20 <genexpr> _bytecode_overflow_example.py CALL \n'\n '20 <genexpr> _bytecode_overflow_example.py line \n'\n '20 <genexpr> _bytecode_overflow_example.py RETURN ing\n'\n '20 <genexpr> _bytecode_overflow_example.py CALL \n'\n '20 <genexpr> _bytecode_overflow_example.py line \n'\n '20 <genexpr> _bytecode_overflow_example.py RETURN \n'\n '21 fun _bytecode_overflow_example.py line \n'\n '22 fun _bytecode_overflow_example.py line \n'\n '23 fun _bytecode_overflow_example.py line \n'\n '24 fun _bytecode_overflow_example.py line \n'\n '25 fun _bytecode_overflow_example.py line \n'\n '24 fun _bytecode_overflow_example.py line \n'\n '26 fun _bytecode_overflow_example.py line \n'\n '24 fun _bytecode_overflow_example.py line \n'\n '27 fun _bytecode_overflow_example.py line \n'\n '24 fun _bytecode_overflow_example.py line \n'\n '27 fun _bytecode_overflow_example.py line \n'\n '24 fun _bytecode_overflow_example.py line \n'\n '28 fun _bytecode_overflow_example.py line \n'\n '24 fun _bytecode_overflow_example.py line \n'\n '29 fun _bytecode_overflow_example.py line \n'\n '24 fun _bytecode_overflow_example.py line \n'\n '30 fun _bytecode_overflow_example.py line \n'\n '24 fun _bytecode_overflow_example.py line \n'\n '31 fun _bytecode_overflow_example.py line \n'\n '24 fun _bytecode_overflow_example.py line \n'\n '31 fun _bytecode_overflow_example.py line \n'\n '24 fun _bytecode_overflow_example.py line \n'\n '34 fun _bytecode_overflow_example.py line \n'\n '35 fun _bytecode_overflow_example.py line \n'\n '36 fun _bytecode_overflow_example.py line \n'\n '37 fun _bytecode_overflow_example.py line \n'\n '38 fun _bytecode_overflow_example.py line \n'\n '22 fun _bytecode_overflow_example.py line \n'\n '22 fun _bytecode_overflow_example.py RETURN \n') == ('18 fun _bytecode_overflow_example.py CALL \n'\n '20 fun _bytecode_overflow_example.py line \n'\n '20 <genexpr> _bytecode_overflow_example.py CALL \n'\n '20 <genexpr> _bytecode_overflow_example.py line \n'\n '20 <genexpr> _bytecode_overflow_example.py RETURN ing\n'\n '20 <genexpr> _bytecode_overflow_example.py CALL \n'\n '20 <genexpr> _bytecode_overflow_example.py line \n'\n '20 <genexpr> _bytecode_overflow_example.py RETURN \n'\n '21 fun _bytecode_overflow_example.py line \n'\n '22 fun _bytecode_overflow_example.py line \n'\n '23 fun _bytecode_overflow_example.py line \n'\n '24 fun _bytecode_overflow_example.py line \n'\n '25 fun _bytecode_overflow_example.py line \n'\n '24 fun _bytecode_overflow_example.py line \n'\n '26 fun _bytecode_overflow_example.py line \n'\n '24 fun _bytecode_overflow_example.py line \n'\n '27 fun _bytecode_overflow_example.py line \n'\n '24 fun _bytecode_overflow_example.py line \n'\n '27 fun _bytecode_overflow_example.py line \n'\n '28 fun _bytecode_overflow_example.py line \n'\n '29 fun _bytecode_overflow_example.py line \n'\n '24 fun _bytecode_overflow_example.py line \n'\n '30 fun _bytecode_overflow_example.py line \n'\n '24 fun _bytecode_overflow_example.py line \n'\n '31 fun _bytecode_overflow_example.py line \n'\n '24 fun _bytecode_overflow_example.py line \n'\n '31 fun _bytecode_overflow_example.py line \n'\n '34 fun _bytecode_overflow_example.py line \n'\n '35 fun _bytecode_overflow_example.py line \n'\n '36 fun _bytecode_overflow_example.py line \n'\n '37 fun _bytecode_overflow_example.py line \n'\n '38 fun _bytecode_overflow_example.py line \n'\n '22 fun _bytecode_overflow_example.py line \n'\n '22 fun _bytecode_overflow_example.py RETURN \n')
E 18 fun _bytecode_overflow_example.py CALL
E 20 fun _bytecode_overflow_example.py line
E 20 <genexpr> _bytecode_overflow_example.py CALL
E 20 <genexpr> _bytecode_overflow_example.py line
E 20 <genexpr> _bytecode_overflow_example.py RETURN ing
E 20 <genexpr> _bytecode_overflow_example.py CALL
E 20 <genexpr> _bytecode_overflow_example.py line
E 20 <genexpr> _bytecode_overflow_example.py RETURN
E 21 fun _bytecode_overflow_example.py line
E 22 fun _bytecode_overflow_example.py line
E 23 fun _bytecode_overflow_example.py line
E 24 fun _bytecode_overflow_example.py line
E 25 fun _bytecode_overflow_example.py line
E 24 fun _bytecode_overflow_example.py line
E 26 fun _bytecode_overflow_example.py line
E 24 fun _bytecode_overflow_example.py line
E 27 fun _bytecode_overflow_example.py line
E 24 fun _bytecode_overflow_example.py line
E 27 fun _bytecode_overflow_example.py line
E + 24 fun _bytecode_overflow_example.py line
E 28 fun _bytecode_overflow_example.py line
E + 24 fun _bytecode_overflow_example.py line
E 29 fun _bytecode_overflow_example.py line
E 24 fun _bytecode_overflow_example.py line
E 30 fun _bytecode_overflow_example.py line
E 24 fun _bytecode_overflow_example.py line
E 31 fun _bytecode_overflow_example.py line
E 24 fun _bytecode_overflow_example.py line
E 31 fun _bytecode_overflow_example.py line
E + 24 fun _bytecode_overflow_example.py line
E 34 fun _bytecode_overflow_example.py line
E 35 fun _bytecode_overflow_example.py line
E 36 fun _bytecode_overflow_example.py line
E 37 fun _bytecode_overflow_example.py line
E 38 fun _bytecode_overflow_example.py line
E 22 fun _bytecode_overflow_example.py line
E 22 fun _bytecode_overflow_example.py RETURN
tests_python/test_bytecode_manipulation.py:174: AssertionError
----------------------------- Captured stdout call -----------------------------
------- replacement at line: 20 ---------
------- baseline ---------
18 fun _bytecode_overflow_example.py CALL
20 fun _bytecode_overflow_example.py line
20 <genexpr> _bytecode_overflow_example.py CALL
20 <genexpr> _bytecode_overflow_example.py line
20 <genexpr> _bytecode_overflow_example.py RETURN ing
20 <genexpr> _bytecode_overflow_example.py CALL
20 <genexpr> _bytecode_overflow_example.py line
20 <genexpr> _bytecode_overflow_example.py RETURN
21 fun _bytecode_overflow_example.py line
22 fun _bytecode_overflow_example.py line
23 fun _bytecode_overflow_example.py line
24 fun _bytecode_overflow_example.py line
25 fun _bytecode_overflow_example.py line
24 fun _bytecode_overflow_example.py line
26 fun _bytecode_overflow_example.py line
24 fun _bytecode_overflow_example.py line
27 fun _bytecode_overflow_example.py line
24 fun _bytecode_overflow_example.py line
27 fun _bytecode_overflow_example.py line
24 fun _bytecode_overflow_example.py line
28 fun _bytecode_overflow_example.py line
24 fun _bytecode_overflow_example.py line
29 fun _bytecode_overflow_example.py line
24 fun _bytecode_overflow_example.py line
30 fun _bytecode_overflow_example.py line
24 fun _bytecode_overflow_example.py line
31 fun _bytecode_overflow_example.py line
24 fun _bytecode_overflow_example.py line
31 fun _bytecode_overflow_example.py line
24 fun _bytecode_overflow_example.py line
34 fun _bytecode_overflow_example.py line
35 fun _bytecode_overflow_example.py line
36 fun _bytecode_overflow_example.py line
37 fun _bytecode_overflow_example.py line
38 fun _bytecode_overflow_example.py line
22 fun _bytecode_overflow_example.py line
22 fun _bytecode_overflow_example.py RETURN
------- contents ---------
18 fun _bytecode_overflow_example.py CALL
20 fun _bytecode_overflow_example.py line
20 <genexpr> _bytecode_overflow_example.py CALL
20 <genexpr> _bytecode_overflow_example.py line
20 <genexpr> _bytecode_overflow_example.py RETURN ing
20 <genexpr> _bytecode_overflow_example.py CALL
20 <genexpr> _bytecode_overflow_example.py line
20 <genexpr> _bytecode_overflow_example.py RETURN
21 fun _bytecode_overflow_example.py line
22 fun _bytecode_overflow_example.py line
23 fun _bytecode_overflow_example.py line
24 fun _bytecode_overflow_example.py line
25 fun _bytecode_overflow_example.py line
24 fun _bytecode_overflow_example.py line
26 fun _bytecode_overflow_example.py line
24 fun _bytecode_overflow_example.py line
27 fun _bytecode_overflow_example.py line
24 fun _bytecode_overflow_example.py line
27 fun _bytecode_overflow_example.py line
28 fun _bytecode_overflow_example.py line
29 fun _bytecode_overflow_example.py line
24 fun _bytecode_overflow_example.py line
30 fun _bytecode_overflow_example.py line
24 fun _bytecode_overflow_example.py line
31 fun _bytecode_overflow_example.py line
24 fun _bytecode_overflow_example.py line
31 fun _bytecode_overflow_example.py line
34 fun _bytecode_overflow_example.py line
35 fun _bytecode_overflow_example.py line
36 fun _bytecode_overflow_example.py line
37 fun _bytecode_overflow_example.py line
38 fun _bytecode_overflow_example.py line
22 fun _bytecode_overflow_example.py line
22 fun _bytecode_overflow_example.py RETURN
-------- error -----------
=============================== warnings summary ===============================
../../../../../../usr/lib/python3/dist-packages/pytest_asyncio/plugin.py:191
/usr/lib/python3/dist-packages/pytest_asyncio/plugin.py:191: DeprecationWarning: The 'asyncio_mode' default value will change to 'strict' in future, please explicitly use 'asyncio_mode=strict' or 'asyncio_mode=auto' in pytest configuration file.
config.issue_config_time_warning(LEGACY_MODE, stacklevel=2)
-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
=========================== short test summary info ============================
FAILED tests_python/test_bytecode_manipulation.py::test_set_pydevd_break_01
========================= 1 failed, 1 warning in 0.19s =========================
(pydevd-venv) euler:~/debian/spyder-packages/pydevd/PyDev.Debugger-testing (main) $ PYDEVD_USE_CYTHON=YES pytest tests_python/test_bytecode_manipulation.py::test_set_pydevd_break_01a
============================= test session starts ==============================
platform linux -- Python 3.10.5, pytest-7.1.2, pluggy-1.0.0+repack -- /usr/bin/python3
PYDEVD_USE_CYTHON: True
PYDEVD_TEST_VM: None
Number of processors: 32
Relevant system paths:
sys.executable: /usr/bin/python3
sys.prefix: /usr
sys.base_prefix: /usr
site.getusersitepackages(): /home/jdg/.local/lib/python3.10/site-packages
site.getsitepackages(): ['/usr/local/lib/python3.10/dist-packages', '/usr/lib/python3/dist-packages', '/usr/lib/python3.10/dist-packages']
Folder with "site-packages" in sys.path: /home/jdg/.local/lib/python3.10/site-packages
cachedir: .pytest_cache
PyQt5 5.15.7 -- Qt runtime 5.15.4 -- Qt compiled 5.15.4
hypothesis profile 'default' -> database=DirectoryBasedExampleDatabase('/home/jdg/debian/spyder-packages/pydevd/PyDev.Debugger-testing/.hypothesis/examples')
rootdir: /home/jdg/debian/spyder-packages/pydevd/PyDev.Debugger-testing, configfile: pytest.ini
plugins: cov-3.0.0, dependency-0.5.1, xdist-2.5.0, mock-3.8.2, forked-1.4.0, qt-4.0.2, flaky-3.7.0, anyio-3.6.1, order-1.0.1, timeout-2.1.0, asyncio-0.18.3, hypothesis-6.36.0, lazy-fixture-0.6.3, xvfb-2.0.0
asyncio: mode=legacy
collected 1 item
tests_python/test_bytecode_manipulation.py::test_set_pydevd_break_01a FAILED [100%]
=================================== FAILURES ===================================
__________________________ test_set_pydevd_break_01a ___________________________
def test_set_pydevd_break_01a():
from tests_python.resources import _bytecode_overflow_example
> check('_bytecode_overflow_example.py', _bytecode_overflow_example.check_backtrack, method_kwargs={'x': 'f'})
tests_python/test_bytecode_manipulation.py:191:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
filename = '_bytecode_overflow_example.py'
method = <function check_backtrack at 0x7fea0dcbb010>
method_kwargs = {'x': 'f'}, skip_breaks_at_lines = set()
method_to_change = <function check_backtrack at 0x7fea0dcbb010>
stop_at_all_lines = False, has_line_event_optimized_in_original_case = False
def check(
filename,
method,
method_kwargs=None,
skip_breaks_at_lines=None,
method_to_change=None,
stop_at_all_lines=False,
has_line_event_optimized_in_original_case=False,
):
'''
:param has_line_event_optimized_in_original_case:
If True, we're handling a case where we have a double jump, i.e.: some case
where there's a JUMP_FORWARD which points to a JUMP_ABSOLUTE and this is
optimized so that the JUMP_FORWARD is changed directly to a JUMP_ABSOLUTE and
we end up skipping one line event which is supposed to be there but isn't in
the initial case but appears when we run after modifying the bytecode in memory.
See: https://github.com/microsoft/debugpy/issues/973#issuecomment-1178090731
'''
from _pydevd_frame_eval.pydevd_modify_bytecode import _get_code_line_info
from _pydevd_frame_eval import pydevd_modify_bytecode
if method_to_change is None:
method_to_change = method
if method_kwargs is None:
method_kwargs = {}
if skip_breaks_at_lines is None:
skip_breaks_at_lines = set()
pydev_break_stops = []
def _pydev_needs_stop_at_break(line):
pydev_break_stops.append(line)
return False
tracer = _Tracer()
def accept_frame(f):
return filename in f.f_code.co_filename
code = method_to_change.__code__
code_line_info = _get_code_line_info(code)
try:
tracer.accept_frame = accept_frame
def call():
method(**method_kwargs)
tracer.call(call)
breakpoint_hit_at_least_once = False
# Ok, we just ran the tracer once without any breakpoints.
#
# Gather its tracing profile: this will be our baseline for further tests (it should contain
# the events and the order in which the were executed).
#
# Note: methods cannot have random elements when executing (otherwise
# the order would be different and the test would be expected to fail).
baseline = tracer.stream.getvalue()
for line in sorted(code_line_info.line_to_offset):
if line in skip_breaks_at_lines:
continue
# Now, for each valid line, add a breakpoint and check if the tracing profile is exactly
# the same (and if the line where we added the breakpoint was executed, see if our
# callback got called).
success, new_code = pydevd_modify_bytecode.insert_pydevd_breaks(code, set([line]), _pydev_needs_stop_at_break=_pydev_needs_stop_at_break)
assert success
method_to_change.__code__ = new_code
tracer = _Tracer()
tracer.accept_frame = accept_frame
tracer.call(call)
contents = tracer.stream.getvalue()
assert tracer.lines_executed
if has_line_event_optimized_in_original_case:
lines = sorted(set(x[1] for x in dis.findlinestarts(new_code)))
new_line_contents = []
last_line = str(max(lines)) + ' '
for l in contents.splitlines(keepends=True):
if not l.strip().startswith(last_line):
new_line_contents.append(l)
contents = ''.join(new_line_contents)
if line in tracer.lines_executed:
assert set([line]) == set(pydev_break_stops)
breakpoint_hit_at_least_once = True
else:
if stop_at_all_lines:
raise AssertionError('Expected the debugger to stop at all lines. Did not stop at line: %s' % (line,))
del pydev_break_stops[:]
if baseline != contents:
print('------- replacement at line: %s ---------' % (line,))
print('------- baseline ---------')
print(baseline)
print('------- contents ---------')
print(contents)
print('-------- error -----------')
> assert baseline == contents
E AssertionError: assert ('1 check_backtrack _bytecode_overflow_example.py CALL \n'\n '2 check_backtrack _bytecode_overflow_example.py line \n'\n '3 check_backtrack _bytecode_overflow_example.py line \n'\n '2 check_backtrack _bytecode_overflow_example.py line \n'\n '4 check_backtrack _bytecode_overflow_example.py line \n'\n '4 check_backtrack _bytecode_overflow_example.py RETURN \n') == ('1 check_backtrack _bytecode_overflow_example.py CALL \n'\n '2 check_backtrack _bytecode_overflow_example.py line \n'\n '3 check_backtrack _bytecode_overflow_example.py line \n'\n '4 check_backtrack _bytecode_overflow_example.py line \n'\n '4 check_backtrack _bytecode_overflow_example.py RETURN \n')
E 1 check_backtrack _bytecode_overflow_example.py CALL
E 2 check_backtrack _bytecode_overflow_example.py line
E 3 check_backtrack _bytecode_overflow_example.py line
E + 2 check_backtrack _bytecode_overflow_example.py line
E 4 check_backtrack _bytecode_overflow_example.py line
E 4 check_backtrack _bytecode_overflow_example.py RETURN
tests_python/test_bytecode_manipulation.py:174: AssertionError
----------------------------- Captured stdout call -----------------------------
------- replacement at line: 2 ---------
------- baseline ---------
1 check_backtrack _bytecode_overflow_example.py CALL
2 check_backtrack _bytecode_overflow_example.py line
3 check_backtrack _bytecode_overflow_example.py line
2 check_backtrack _bytecode_overflow_example.py line
4 check_backtrack _bytecode_overflow_example.py line
4 check_backtrack _bytecode_overflow_example.py RETURN
------- contents ---------
1 check_backtrack _bytecode_overflow_example.py CALL
2 check_backtrack _bytecode_overflow_example.py line
3 check_backtrack _bytecode_overflow_example.py line
4 check_backtrack _bytecode_overflow_example.py line
4 check_backtrack _bytecode_overflow_example.py RETURN
-------- error -----------
=============================== warnings summary ===============================
../../../../../../usr/lib/python3/dist-packages/pytest_asyncio/plugin.py:191
/usr/lib/python3/dist-packages/pytest_asyncio/plugin.py:191: DeprecationWarning: The 'asyncio_mode' default value will change to 'strict' in future, please explicitly use 'asyncio_mode=strict' or 'asyncio_mode=auto' in pytest configuration file.
config.issue_config_time_warning(LEGACY_MODE, stacklevel=2)
-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
=========================== short test summary info ============================
FAILED tests_python/test_bytecode_manipulation.py::test_set_pydevd_break_01a
========================= 1 failed, 1 warning in 0.17s =========================[EDIT: Oops; I accidentally used the system's pytest rather than the venv one; rerunning with python -m pytest gave the same results, though.]
juliangilbey
commented
1 year ago Ah, these two tests now work; clearly the patch has either been updated or something else has changed to make it work. Closing this issue. Thanks!!
civodul
commented
1 year ago Hi! I still see the same test failure with 2.9.6 running on Python 3.10.7:
platform linux -- Python 3.10.7, pytest-7.1.3, pluggy-1.0.0 -- /gnu/store/l6fpy0i9hlll9b6k8vy2i2a4cshwz3cv-python-wrapper-3.10.7/bin/python
PYDEVD_USE_CYTHON: True
PYDEVD_TEST_VM: None
[...]
=================================== FAILURES ===================================
__________________________ test_set_pydevd_break_01a ___________________________
[gw2] linux -- Python 3.10.7 /gnu/store/l6fpy0i9hlll9b6k8vy2i2a4cshwz3cv-python-wrapper-3.10.7/bin/python
def test_set_pydevd_break_01a():
from tests_python.resources import _bytecode_overflow_example
> check('_bytecode_overflow_example.py', _bytecode_overflow_example.check_backtrack, method_kwargs={'x': 'f'})
tests_python/test_bytecode_manipulation.py:191:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
filename = '_bytecode_overflow_example.py'
method = <function check_backtrack at 0x7ffff4e33490>
method_kwargs = {'x': 'f'}, skip_breaks_at_lines = set()
method_to_change = <function check_backtrack at 0x7ffff4e33490>
stop_at_all_lines = False, has_line_event_optimized_in_original_case = False
def check(
filename,
method,
method_kwargs=None,
skip_breaks_at_lines=None,
method_to_change=None,
stop_at_all_lines=False,
has_line_event_optimized_in_original_case=False,
):
'''
:param has_line_event_optimized_in_original_case:
If True, we're handling a case where we have a double jump, i.e.: some case
where there's a JUMP_FORWARD which points to a JUMP_ABSOLUTE and this is
optimized so that the JUMP_FORWARD is changed directly to a JUMP_ABSOLUTE and
we end up skipping one line event which is supposed to be there but isn't in
the initial case but appears when we run after modifying the bytecode in memory.
See: https://github.com/microsoft/debugpy/issues/973#issuecomment-1178090731
'''
from _pydevd_frame_eval.pydevd_modify_bytecode import _get_code_line_info
from _pydevd_frame_eval import pydevd_modify_bytecode
if method_to_change is None:
method_to_change = method
if method_kwargs is None:
method_kwargs = {}
if skip_breaks_at_lines is None:
skip_breaks_at_lines = set()
pydev_break_stops = []
def _pydev_needs_stop_at_break(line):
pydev_break_stops.append(line)
return False
tracer = _Tracer()
def accept_frame(f):
return filename in f.f_code.co_filename
code = method_to_change.__code__
code_line_info = _get_code_line_info(code)
try:
tracer.accept_frame = accept_frame
def call():
method(**method_kwargs)
tracer.call(call)
breakpoint_hit_at_least_once = False
# Ok, we just ran the tracer once without any breakpoints.
#
# Gather its tracing profile: this will be our baseline for further tests (it should contain
# the events and the order in which the were executed).
#
# Note: methods cannot have random elements when executing (otherwise
# the order would be different and the test would be expected to fail).
baseline = tracer.stream.getvalue()
for line in sorted(code_line_info.line_to_offset):
if line in skip_breaks_at_lines:
continue
# Now, for each valid line, add a breakpoint and check if the tracing profile is exactly
# the same (and if the line where we added the breakpoint was executed, see if our
# callback got called).
success, new_code = pydevd_modify_bytecode.insert_pydevd_breaks(code, set([line]), _pydev_needs_stop_at_break=_pydev_needs_stop_at_break)
assert success
method_to_change.__code__ = new_code
tracer = _Tracer()
tracer.accept_frame = accept_frame
tracer.call(call)
contents = tracer.stream.getvalue()
assert tracer.lines_executed
if has_line_event_optimized_in_original_case:
lines = sorted(set(x[1] for x in dis.findlinestarts(new_code)))
new_line_contents = []
last_line = str(max(lines)) + ' '
for l in contents.splitlines(keepends=True):
if not l.strip().startswith(last_line):
new_line_contents.append(l)
contents = ''.join(new_line_contents)
if line in tracer.lines_executed:
assert set([line]) == set(pydev_break_stops)
breakpoint_hit_at_least_once = True
else:
if stop_at_all_lines:
raise AssertionError('Expected the debugger to stop at all lines. Did not stop at line: %s' % (line,))
del pydev_break_stops[:]
if baseline != contents:
print('------- replacement at line: %s ---------' % (line,))
print('------- baseline ---------')
print(baseline)
print('------- contents ---------')
print(contents)
print('-------- error -----------')
> assert baseline == contents
E AssertionError: assert ('1 check_backtrack _bytecode_overflow_example.py CALL \n'\n '2 check_backtrack _bytecode_overflow_example.py line \n'\n '3 check_backtrack _bytecode_overflow_example.py line \n'\n '2 check_backtrack _bytecode_overflow_example.py line \n'\n '4 check_backtrack _bytecode_overflow_example.py line \n'\n '4 check_backtrack _bytecode_overflow_example.py RETURN \n') == ('1 check_backtrack _bytecode_overflow_example.py CALL \n'\n '2 check_backtrack _bytecode_overflow_example.py line \n'\n '3 check_backtrack _bytecode_overflow_example.py line \n'\n '4 check_backtrack _bytecode_overflow_example.py line \n'\n '4 check_backtrack _bytecode_overflow_example.py RETURN \n')
E 1 check_backtrack _bytecode_overflow_example.py CALL
E 2 check_backtrack _bytecode_overflow_example.py line
E 3 check_backtrack _bytecode_overflow_example.py line
E + 2 check_backtrack _bytecode_overflow_example.py line
E 4 check_backtrack _bytecode_overflow_example.py line
E 4 check_backtrack _bytecode_overflow_example.py RETURN
tests_python/test_bytecode_manipulation.py:174: AssertionError
----------------------------- Captured stdout call -----------------------------
------- replacement at line: 2 ---------
------- baseline ---------
1 check_backtrack _bytecode_overflow_example.py CALL
2 check_backtrack _bytecode_overflow_example.py line
3 check_backtrack _bytecode_overflow_example.py line
2 check_backtrack _bytecode_overflow_example.py line
4 check_backtrack _bytecode_overflow_example.py line
4 check_backtrack _bytecode_overflow_example.py RETURN
------- contents ---------
1 check_backtrack _bytecode_overflow_example.py CALL
2 check_backtrack _bytecode_overflow_example.py line
3 check_backtrack _bytecode_overflow_example.py line
4 check_backtrack _bytecode_overflow_example.py line
4 check_backtrack _bytecode_overflow_example.py RETURN
-------- error -----------
___________________________ test_set_pydevd_break_01 ___________________________
[gw1] linux -- Python 3.10.7 /gnu/store/l6fpy0i9hlll9b6k8vy2i2a4cshwz3cv-python-wrapper-3.10.7/bin/python
def test_set_pydevd_break_01():
from tests_python.resources import _bytecode_overflow_example
> check('_bytecode_overflow_example.py', _bytecode_overflow_example.Dummy.fun, method_kwargs={'text': 'ing'}, has_line_event_optimized_in_original_case=True)
tests_python/test_bytecode_manipulation.py:185:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
filename = '_bytecode_overflow_example.py'
method = <function Dummy.fun at 0x7ffff513fe20>, method_kwargs = {'text': 'ing'}
skip_breaks_at_lines = set()
method_to_change = <function Dummy.fun at 0x7ffff513fe20>
stop_at_all_lines = False, has_line_event_optimized_in_original_case = True
def check(
filename,
method,
method_kwargs=None,
skip_breaks_at_lines=None,
method_to_change=None,
stop_at_all_lines=False,
has_line_event_optimized_in_original_case=False,
):
'''
:param has_line_event_optimized_in_original_case:
If True, we're handling a case where we have a double jump, i.e.: some case
where there's a JUMP_FORWARD which points to a JUMP_ABSOLUTE and this is
optimized so that the JUMP_FORWARD is changed directly to a JUMP_ABSOLUTE and
we end up skipping one line event which is supposed to be there but isn't in
the initial case but appears when we run after modifying the bytecode in memory.
See: https://github.com/microsoft/debugpy/issues/973#issuecomment-1178090731
'''
from _pydevd_frame_eval.pydevd_modify_bytecode import _get_code_line_info
from _pydevd_frame_eval import pydevd_modify_bytecode
if method_to_change is None:
method_to_change = method
if method_kwargs is None:
method_kwargs = {}
if skip_breaks_at_lines is None:
skip_breaks_at_lines = set()
pydev_break_stops = []
def _pydev_needs_stop_at_break(line):
pydev_break_stops.append(line)
return False
tracer = _Tracer()
def accept_frame(f):
return filename in f.f_code.co_filename
code = method_to_change.__code__
code_line_info = _get_code_line_info(code)
try:
tracer.accept_frame = accept_frame
def call():
method(**method_kwargs)
tracer.call(call)
breakpoint_hit_at_least_once = False
# Ok, we just ran the tracer once without any breakpoints.
#
# Gather its tracing profile: this will be our baseline for further tests (it should contain
# the events and the order in which the were executed).
#
# Note: methods cannot have random elements when executing (otherwise
# the order would be different and the test would be expected to fail).
baseline = tracer.stream.getvalue()
for line in sorted(code_line_info.line_to_offset):
if line in skip_breaks_at_lines:
continue
# Now, for each valid line, add a breakpoint and check if the tracing profile is exactly
# the same (and if the line where we added the breakpoint was executed, see if our
# callback got called).
success, new_code = pydevd_modify_bytecode.insert_pydevd_breaks(code, set([line]), _pydev_needs_stop_at_break=_pydev_needs_stop_at_break)
assert success
method_to_change.__code__ = new_code
tracer = _Tracer()
tracer.accept_frame = accept_frame
tracer.call(call)
contents = tracer.stream.getvalue()
assert tracer.lines_executed
if has_line_event_optimized_in_original_case:
lines = sorted(set(x[1] for x in dis.findlinestarts(new_code)))
new_line_contents = []
last_line = str(max(lines)) + ' '
for l in contents.splitlines(keepends=True):
if not l.strip().startswith(last_line):
new_line_contents.append(l)
contents = ''.join(new_line_contents)
if line in tracer.lines_executed:
assert set([line]) == set(pydev_break_stops)
breakpoint_hit_at_least_once = True
else:
if stop_at_all_lines:
raise AssertionError('Expected the debugger to stop at all lines. Did not stop at line: %s' % (line,))
del pydev_break_stops[:]
if baseline != contents:
print('------- replacement at line: %s ---------' % (line,))
print('------- baseline ---------')
print(baseline)
print('------- contents ---------')
print(contents)
print('-------- error -----------')
> assert baseline == contents
E AssertionError: assert ('18 fun _bytecode_overflow_example.py CALL \n'\n '20 fun _bytecode_overflow_example.py line \n'\n '20 <genexpr> _bytecode_overflow_example.py CALL \n'\n '20 <genexpr> _bytecode_overflow_example.py line \n'\n '20 <genexpr> _bytecode_overflow_example.py RETURN ing\n'\n '20 <genexpr> _bytecode_overflow_example.py CALL \n'\n '20 <genexpr> _bytecode_overflow_example.py line \n'\n '20 <genexpr> _bytecode_overflow_example.py RETURN \n'\n '21 fun _bytecode_overflow_example.py line \n'\n '22 fun _bytecode_overflow_example.py line \n'\n '23 fun _bytecode_overflow_example.py line \n'\n '24 fun _bytecode_overflow_example.py line \n'\n '25 fun _bytecode_overflow_example.py line \n'\n '24 fun _bytecode_overflow_example.py line \n'\n '26 fun _bytecode_overflow_example.py line \n'\n '24 fun _bytecode_overflow_example.py line \n'\n '27 fun _bytecode_overflow_example.py line \n'\n '24 fun _bytecode_overflow_example.py line \n'\n '27 fun _bytecode_overflow_example.py line \n'\n '24 fun _bytecode_overflow_example.py line \n'\n '28 fun _bytecode_overflow_example.py line \n'\n '24 fun _bytecode_overflow_example.py line \n'\n '29 fun _bytecode_overflow_example.py line \n'\n '24 fun _bytecode_overflow_example.py line \n'\n '30 fun _bytecode_overflow_example.py line \n'\n '24 fun _bytecode_overflow_example.py line \n'\n '31 fun _bytecode_overflow_example.py line \n'\n '24 fun _bytecode_overflow_example.py line \n'\n '31 fun _bytecode_overflow_example.py line \n'\n '24 fun _bytecode_overflow_example.py line \n'\n '34 fun _bytecode_overflow_example.py line \n'\n '35 fun _bytecode_overflow_example.py line \n'\n '36 fun _bytecode_overflow_example.py line \n'\n '37 fun _bytecode_overflow_example.py line \n'\n '38 fun _bytecode_overflow_example.py line \n'\n '22 fun _bytecode_overflow_example.py line \n'\n '22 fun _bytecode_overflow_example.py RETURN \n') == ('18 fun _bytecode_overflow_example.py CALL \n'\n '20 fun _bytecode_overflow_example.py line \n'\n '20 <genexpr> _bytecode_overflow_example.py CALL \n'\n '20 <genexpr> _bytecode_overflow_example.py line \n'\n '20 <genexpr> _bytecode_overflow_example.py RETURN ing\n'\n '20 <genexpr> _bytecode_overflow_example.py CALL \n'\n '20 <genexpr> _bytecode_overflow_example.py line \n'\n '20 <genexpr> _bytecode_overflow_example.py RETURN \n'\n '21 fun _bytecode_overflow_example.py line \n'\n '22 fun _bytecode_overflow_example.py line \n'\n '23 fun _bytecode_overflow_example.py line \n'\n '24 fun _bytecode_overflow_example.py line \n'\n '25 fun _bytecode_overflow_example.py line \n'\n '24 fun _bytecode_overflow_example.py line \n'\n '26 fun _bytecode_overflow_example.py line \n'\n '24 fun _bytecode_overflow_example.py line \n'\n '27 fun _bytecode_overflow_example.py line \n'\n '24 fun _bytecode_overflow_example.py line \n'\n '27 fun _bytecode_overflow_example.py line \n'\n '28 fun _bytecode_overflow_example.py line \n'\n '29 fun _bytecode_overflow_example.py line \n'\n '24 fun _bytecode_overflow_example.py line \n'\n '30 fun _bytecode_overflow_example.py line \n'\n '24 fun _bytecode_overflow_example.py line \n'\n '31 fun _bytecode_overflow_example.py line \n'\n '24 fun _bytecode_overflow_example.py line \n'\n '31 fun _bytecode_overflow_example.py line \n'\n '34 fun _bytecode_overflow_example.py line \n'\n '35 fun _bytecode_overflow_example.py line \n'\n '36 fun _bytecode_overflow_example.py line \n'\n '37 fun _bytecode_overflow_example.py line \n'\n '38 fun _bytecode_overflow_example.py line \n'\n '22 fun _bytecode_overflow_example.py line \n'\n '22 fun _bytecode_overflow_example.py RETURN \n')
E 18 fun _bytecode_overflow_example.py CALL
E 20 fun _bytecode_overflow_example.py line
E 20 <genexpr> _bytecode_overflow_example.py CALL
E 20 <genexpr> _bytecode_overflow_example.py line
E 20 <genexpr> _bytecode_overflow_example.py RETURN ing
E 20 <genexpr> _bytecode_overflow_example.py CALL
E 20 <genexpr> _bytecode_overflow_example.py line
E 20 <genexpr> _bytecode_overflow_example.py RETURN
E 21 fun _bytecode_overflow_example.py line
E 22 fun _bytecode_overflow_example.py line
E 23 fun _bytecode_overflow_example.py line
E 24 fun _bytecode_overflow_example.py line
E 25 fun _bytecode_overflow_example.py line
E 24 fun _bytecode_overflow_example.py line
E 26 fun _bytecode_overflow_example.py line
E 24 fun _bytecode_overflow_example.py line
E 27 fun _bytecode_overflow_example.py line
E 24 fun _bytecode_overflow_example.py line
E 27 fun _bytecode_overflow_example.py line
E + 24 fun _bytecode_overflow_example.py line
E 28 fun _bytecode_overflow_example.py line
E + 24 fun _bytecode_overflow_example.py line
E 29 fun _bytecode_overflow_example.py line
E 24 fun _bytecode_overflow_example.py line
E 30 fun _bytecode_overflow_example.py line
E 24 fun _bytecode_overflow_example.py line
E 31 fun _bytecode_overflow_example.py line
E 24 fun _bytecode_overflow_example.py line
E 31 fun _bytecode_overflow_example.py line
E + 24 fun _bytecode_overflow_example.py line
E 34 fun _bytecode_overflow_example.py line
E 35 fun _bytecode_overflow_example.py line
E 36 fun _bytecode_overflow_example.py line
E 37 fun _bytecode_overflow_example.py line
E 38 fun _bytecode_overflow_example.py line
E 22 fun _bytecode_overflow_example.py line
E 22 fun _bytecode_overflow_example.py RETURN
tests_python/test_bytecode_manipulation.py:174: AssertionError
----------------------------- Captured stdout call -----------------------------
------- replacement at line: 20 ---------
------- baseline ---------
18 fun _bytecode_overflow_example.py CALL
20 fun _bytecode_overflow_example.py line
20 <genexpr> _bytecode_overflow_example.py CALL
20 <genexpr> _bytecode_overflow_example.py line
20 <genexpr> _bytecode_overflow_example.py RETURN ing
20 <genexpr> _bytecode_overflow_example.py CALL
20 <genexpr> _bytecode_overflow_example.py line
20 <genexpr> _bytecode_overflow_example.py RETURN
21 fun _bytecode_overflow_example.py line
22 fun _bytecode_overflow_example.py line
23 fun _bytecode_overflow_example.py line
24 fun _bytecode_overflow_example.py line
25 fun _bytecode_overflow_example.py line
24 fun _bytecode_overflow_example.py line
26 fun _bytecode_overflow_example.py line
24 fun _bytecode_overflow_example.py line
27 fun _bytecode_overflow_example.py line
24 fun _bytecode_overflow_example.py line
27 fun _bytecode_overflow_example.py line
24 fun _bytecode_overflow_example.py line
28 fun _bytecode_overflow_example.py line
24 fun _bytecode_overflow_example.py line
29 fun _bytecode_overflow_example.py line
24 fun _bytecode_overflow_example.py line
30 fun _bytecode_overflow_example.py line
24 fun _bytecode_overflow_example.py line
31 fun _bytecode_overflow_example.py line
24 fun _bytecode_overflow_example.py line
31 fun _bytecode_overflow_example.py line
24 fun _bytecode_overflow_example.py line
34 fun _bytecode_overflow_example.py line
35 fun _bytecode_overflow_example.py line
36 fun _bytecode_overflow_example.py line
37 fun _bytecode_overflow_example.py line
38 fun _bytecode_overflow_example.py line
22 fun _bytecode_overflow_example.py line
22 fun _bytecode_overflow_example.py RETURN
------- contents ---------
18 fun _bytecode_overflow_example.py CALL
20 fun _bytecode_overflow_example.py line
20 <genexpr> _bytecode_overflow_example.py CALL
20 <genexpr> _bytecode_overflow_example.py line
20 <genexpr> _bytecode_overflow_example.py RETURN ing
20 <genexpr> _bytecode_overflow_example.py CALL
20 <genexpr> _bytecode_overflow_example.py line
20 <genexpr> _bytecode_overflow_example.py RETURN
21 fun _bytecode_overflow_example.py line
22 fun _bytecode_overflow_example.py line
23 fun _bytecode_overflow_example.py line
24 fun _bytecode_overflow_example.py line
25 fun _bytecode_overflow_example.py line
24 fun _bytecode_overflow_example.py line
26 fun _bytecode_overflow_example.py line
24 fun _bytecode_overflow_example.py line
27 fun _bytecode_overflow_example.py line
24 fun _bytecode_overflow_example.py line
27 fun _bytecode_overflow_example.py line
28 fun _bytecode_overflow_example.py line
29 fun _bytecode_overflow_example.py line
24 fun _bytecode_overflow_example.py line
30 fun _bytecode_overflow_example.py line
24 fun _bytecode_overflow_example.py line
31 fun _bytecode_overflow_example.py line
24 fun _bytecode_overflow_example.py line
31 fun _bytecode_overflow_example.py line
34 fun _bytecode_overflow_example.py line
35 fun _bytecode_overflow_example.py line
36 fun _bytecode_overflow_example.py line
37 fun _bytecode_overflow_example.py line
38 fun _bytecode_overflow_example.py line
22 fun _bytecode_overflow_example.py line
22 fun _bytecode_overflow_example.py RETURN
-------- error -----------
__________________________ test_soft_terminate[True] ___________________________
[gw3] linux -- Python 3.10.7 /gnu/store/l6fpy0i9hlll9b6k8vy2i2a4cshwz3cv-python-wrapper-3.10.7/bin/python
self = <ReaderThread(Test Reader Thread, stopped daemon 140736717739584)>
context_message = 'wait_for_message', timeout = 15
def get_next_message(self, context_message, timeout=None):
if timeout is None:
timeout = self.MESSAGES_TIMEOUT
try:
> msg = self._queue.get(block=True, timeout=timeout)
tests_python/debugger_unittest.py:211:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
[...]
=========================== short test summary info ============================
FAILED tests_python/test_bytecode_manipulation.py::test_set_pydevd_break_01a
FAILED tests_python/test_bytecode_manipulation.py::test_set_pydevd_break_01
FAILED tests_python/test_debugger_json.py::test_soft_terminate[True] - Assert...
====== 3 failed, 732 passed, 96 skipped, 20 warnings in 264.24s (0:04:24) ======
Hi! Using the latest GitHub sources (https://github.com/fabioz/PyDev.Debugger/commit/1b1fb8b8a8d1c1068fecf2e28890a2e764eade71) and running the package tests with Python 3.10 (on a Debian system using a virtual environment, set up as in the GitHub workflows file in the package) gives two failing tests; these pass with Python 3.9, so presumably there is some Python bytecode format change involved. The two failing tests are:
In both cases, the error is that the baseline does not match the contents.
Best wishes, Julian