Debugger Heisenbug where integer comparison appears to fail within a specific context

dddvision commented 4 years ago

Environment data

VS Code version: Version: 1.47.2 Commit: 17299e413d5590b14ab0340ea477cdd86ff13daf Date: 2020-07-15T18:22:15.161Z Electron: 7.3.2 Chrome: 78.0.3904.130 Node.js: 12.8.1 V8: 7.8.279.23-electron.0 OS: Linux x64 5.4.0-42-generic snap
Extension version (available under the Extensions sidebar): ms-python.python v2020.7.96456
OS and version: LSB Version: core-9.20170808ubuntu1-noarch:security-9.20170808ubuntu1-noarch Distributor ID: Ubuntu Description: Ubuntu 18.04.4 LTS Release: 18.04 Codename: bionic
Python version (& distribution if applicable, e.g. Anaconda): Anaconda Python 3.7.7
Type of virtual environment used (N/A | venv | virtualenv | conda | ...): Python 3.7.7 64-bit ('base':conda)
Relevant/affected Python packages and their versions: jsonpickle 1.4.1
numpy 1.18.1
SQLAlchemy 1.3.17
Relevant/affected Python-related VS Code extensions and their versions: ms-python.python v2020.7.96456
Value of the python.languageServer setting: Microsoft

Expected behaviour

When debugging the attached file and stepping into main on line 33 with no arguments, the value of len(sys.argv) is 1, so the comparison N>=2 on line 35 should be False and the debugger should step to the else statement on line 39.

Actual behaviour

The debugger steps to line 36 as if N>=2 were True. It also incorrectly steps to line 42 after comparing N>=3.

Steps to reproduce:

Debug the attached file and step into main on line 33 with no arguments, then see if the debugger steps to line 36 or 39. All settings are default with a fresh install of vscode and the python extension.

I have spent many hours trying to isolate the minimal code required to demonstrate this bug. As far as I can tell, every line of uncommented code in this file is required to demonstrate the problem. Furthermore, adding print statements before or after the bug makes it disappear.

If further evidence of this bug is needed, then please ask and I will provide a .gif.

Logs

Nothing appears in the Output panel. In the terminal, the following appears: cd /home/username ; env /home/username/anaconda3/bin/python /home/username/.vscode/extensions/ms-python.python-2020.7.96456/pythonFiles/lib/python/debugpy/launcher 39939 -- /home/username/vscode_debuger_bug.py 1

Code

import os
import sys
import time
import random
import jsonpickle
import numpy as np
import sqlalchemy as sa
from sqlalchemy.ext.declarative import declarative_base
from sqlalchemy.orm import sessionmaker

Base = declarative_base()

def MyFunction():
    pass

class MyClassA(Base):
    __tablename__ = 'tableA'
    n = sa.Column(sa.BigInteger, primary_key=True)

class MyClassB(Base):
    __tablename__ = 'tableB'
    n = sa.Column(sa.BigInteger, primary_key=True)

class MyClassC(object):
    def __init__(self):
        pass

class MyClassD(object):
    def __init__(self):
        pass

if __name__ == "__main__":
    N = len(sys.argv)  # set breakpoint here and pass no input arguments

    if N >= 2:
        arg1 = sys.argv[1]  # the debugger gets here even though N=1
        print(arg1)
    else:
        arg1 = 'MyString'

    if N >= 3:
        arg2 = int(sys.argv[2])  # the debugger gets here even though N=1
    else:
        arg2 = int(0)

    print(N)  # still N=1
    # print(arg1)  # if you print this then it changes the debugger behavior above

int19h commented 4 years ago

I couldn't readily repro it with these steps, unfortunately. It might be that there's some hidden variable at play here....

Can you check something on your repro? When you step through the program, if you have the Run view open, you should be seeing line numbers in the Call Stack section. Do they reflect the highlighted line in the editor? Or are they correct, but the editor is highlighting the wrong line?

dddvision commented 4 years ago

I checked the line numbers in the Call Stack, and they are correct.

I also recorded a gif animation below. Peek 2020-07-24 10-56

I also updated my sqlalchemy library to the following and confirmed the same result. SQLAlchemy 1.3.18

Other libraries related to sqlalchemy that possibly could modify it are: pyrqlite HEAD sqlalchemy-rqlite HEAD

And, just in case it is helpful, here is the full output of 'pip list':

Click to expand!

``` absl-py 0.9.0 actionlib 1.12.0 anaconda-client 1.7.2 anaconda-navigator 1.9.12 angles 1.9.12 appdirs 1.4.3 arviz 0.7.0 astor 0.8.1 astroid 2.3.3 astropy 4.0 attrs 19.3.0 Automat 20.2.0 autopep8 1.5 autosub 0.3.12 Babel 2.8.0 backcall 0.2.0 backports.functools-lru-cache 1.6.1 backports.tempfile 1.0 backports.weakref 1.0.post1 beautifulsoup4 4.9.1 bleach 3.1.5 bokeh 1.4.0 bondpy 1.8.5 brotlipy 0.7.0 cachetools 4.0.0 camera-calibration 1.15.0 camera-calibration-parsers 1.11.13 catkin 0.7.26 catkin-pkg 0.4.16 certifi 2020.6.20 cffi 1.14.0 cftime 1.1.0 chardet 3.0.4 click 7.1.2 cloudpickle 1.3.0 clyent 1.2.2 colorama 0.4.3 conda 4.8.3 conda-build 3.18.11 conda-package-handling 1.7.0 conda-verify 3.4.2 constantly 15.1.0 contextlib2 0.6.0.post1 controller-manager 0.18.1 controller-manager-msgs 0.18.1 coverage 5.0.3 cryptography 2.9.2 cv-bridge 1.13.0 cycler 0.10.0 Cython 0.29.21 dask 2.12.0 decorator 4.4.2 deepdiff 4.2.0 defusedxml 0.6.0 diagnostic-analysis 1.9.3 diagnostic-common-diagnostics 1.9.3 diagnostic-updater 1.9.3 diff-match-patch 20181111 distributed 2.12.0 distro 1.4.0 docutils 0.16 dynamic-reconfigure 1.6.3 entrypoints 0.3 enum34 1.1.9 fastcache 1.1.0 filelock 3.0.12 flake8 3.7.9 Flask 1.1.1 Flask-Cors 3.0.8 future 0.18.2 gast 0.2.2 gazebo-plugins 2.8.7 gazebo-ros 2.8.7 gencpp 0.6.5 geneus 2.2.6 genlisp 0.4.16 genmsg 0.5.16 gennodejs 2.0.1 genpy 0.6.12 gevent 1.4.0 glob2 0.7 google-api-core 1.16.0 google-api-python-client 1.7.11 google-auth 1.11.2 google-auth-httplib2 0.0.3 google-auth-oauthlib 0.4.1 google-cloud-texttospeech 1.0.1 google-pasta 0.1.8 googleapis-common-protos 1.51.0 graphviz 0.13.2 grpcio 1.27.2 h5py 2.10.0 helpdev 0.6.10 httplib2 0.17.0 hyperlink 19.0.0 hypothesis 5.6.0 idna 2.10 image-geometry 1.13.0 imageio 2.8.0 imagesize 1.2.0 importlib-metadata 1.7.0 incremental 17.5.0 interactive-markers 1.11.5 intervaltree 3.0.2 ipykernel 5.3.3 ipython 7.16.1 ipython-genutils 0.2.0 ipywidgets 7.5.1 isort 4.3.21 jdcal 1.4.1 jedi 0.17.1 jeepney 0.4.3 Jinja2 2.11.2 joblib 0.14.1 joint-state-publisher 1.12.15 json5 0.9.5 jsonpickle 1.4.1 jsonschema 3.2.0 jupyter-client 6.1.6 jupyter-core 4.6.3 jupyterlab 2.1.5 jupyterlab-server 1.2.0 kdl-parser-py 1.13.1 Keras 2.3.1 Keras-Applications 1.0.8 Keras-Preprocessing 1.1.0 keras-segmentation 0.2.0 keyring 21.1.1 kiwisolver 1.2.0 laser-geometry 1.6.5 lazy-object-proxy 1.4.3 libarchive-c 2.9 lief 0.10.1 lxml 4.5.0 Markdown 3.2.1 MarkupSafe 1.1.1 matplotlib 3.2.0 message-filters 1.14.6 mistune 0.8.4 mkl-fft 1.0.15 mkl-random 1.1.0 mkl-service 2.3.0 mock 4.0.1 more-itertools 8.2.0 mpyq 0.2.5 msgpack 1.0.0 msgpack-python 0.5.6 navigator-updater 0.2.1 nbconvert 5.6.1 nbformat 5.0.7 netCDF4 1.5.3 nltk 3.4.5 notebook 6.0.3 numexpr 2.7.1 numpy 1.18.1 numpydoc 0.9.2 oauthlib 3.1.0 olefile 0.46 openpyxl 3.0.3 opt-einsum 3.2.0 ordered-set 3.1.1 packaging 20.4 pandas 1.0.1 pandocfilters 1.4.2 parso 0.7.0 partd 1.1.0 path 13.1.0 path.py 12.4.0 pathlib2 2.3.5 pathtools 0.1.2 pbr 5.4.4 peewee 3.13.1 pexpect 4.8.0 pickleshare 0.7.5 Pillow 7.2.0 pip 20.1.1 pkginfo 1.5.0.1 pluggy 0.13.1 portpicker 1.3.1 progressbar 2.5 prometheus-client 0.8.0 prompt-toolkit 3.0.5 protobuf 3.8.0 protobuf3 0.2.1 psutil 5.7.0 ptyprocess 0.6.0 py 1.8.1 pyasn1 0.4.8 pyasn1-modules 0.2.8 pycodestyle 2.5.0 pycosat 0.6.3 pycparser 2.20 pydocstyle 5.0.2 pydot 1.4.1 pyflakes 2.1.1 pygame 1.9.6 Pygments 2.6.1 PyHamcrest 2.0.2 pylint 2.4.4 pymc3 3.8 pyodbc 4.0.30 pyOpenSSL 19.1.0 pyparsing 2.4.7 PyQt5 5.12.3 PyQt5-sip 12.7.1 PyQtWebEngine 5.12.1 pyreadline 2.1 pyrqlite HEAD pyrsistent 0.16.0 PySC2 3.0.0 PySocks 1.7.1 pysrt 1.1.2 pytest 5.3.5 pytest-astropy 0.8.0 pytest-astropy-header 0.1.2 pytest-cov 2.8.1 pytest-doctestplus 0.5.0 pytest-filter-subpackage 0.1.1 pytest-openfiles 0.4.0 python-dateutil 2.8.1 python-jsonrpc-server 0.3.4 python-language-server 0.31.8 python-qt-binding 0.4.3 pytools 2020.1 pytz 2020.1 pyxdg 0.26 PyYAML 5.3.1 pyzmq 19.0.1 QDarkStyle 2.8 qt-dotgraph 0.4.1 qt-gui 0.4.1 qt-gui-cpp 0.4.1 qt-gui-py-common 0.4.1 QtAwesome 0.7.0 qtconsole 4.7.1 QtPy 1.9.0 requests 2.24.0 requests-oauthlib 1.3.0 resource-retriever 1.12.6 rope 0.16.0 rosbag 1.14.3 rosboost-cfg 1.14.9 rosclean 1.14.9 roscreate 1.14.9 rosdep 0.18.0 rosdistro 0.8.0 rosgraph 1.14.6 roslaunch 1.14.6 roslib 1.14.9 roslint 0.11.2 roslz4 1.14.6 rosmake 1.14.9 rosmaster 1.14.6 rosmsg 1.14.6 rosnode 1.14.6 rosparam 1.14.6 rospkg 1.2.3 rospy 1.14.6 rosservice 1.14.6 rostest 1.14.6 rostopic 1.14.6 rosunit 1.14.9 roswtf 1.14.6 rqt-action 0.4.9 rqt-bag 0.4.12 rqt-bag-plugins 0.4.12 rqt-console 0.4.9 rqt-dep 0.4.9 rqt-graph 0.4.11 rqt-gui 0.5.2 rqt-gui-py 0.5.2 rqt-image-view 0.4.14 rqt-launch 0.4.8 rqt-logger-level 0.4.8 rqt-moveit 0.5.7 rqt-msg 0.4.8 rqt-nav-view 0.5.7 rqt-plot 0.4.9 rqt-pose-view 0.5.8 rqt-publisher 0.4.8 rqt-py-common 0.5.2 rqt-py-console 0.4.8 rqt-reconfigure 0.5.3 rqt-robot-dashboard 0.5.7 rqt-robot-monitor 0.5.9 rqt-robot-steering 0.5.10 rqt-runtime-monitor 0.5.7 rqt-rviz 0.6.0 rqt-service-caller 0.4.8 rqt-shell 0.4.9 rqt-srv 0.4.8 rqt-tf-tree 0.6.0 rqt-top 0.4.8 rqt-topic 0.4.11 rqt-web 0.4.8 rsa 4.0 ruamel-yaml 0.15.87 ruamel.yaml.clib 0.2.0 rviz 1.13.13 s2clientprotocol 4.11.4.78285.0 s2protocol 4.11.4.78285.0 scikit-image 0.16.2 scikit-learn 0.22.2.post1 scipy 1.4.1 seaborn 0.10.0 SecretStorage 3.1.2 Send2Trash 1.5.0 sensor-msgs 1.12.7 service-identity 18.1.0 setuptools 49.2.0.post20200714 six 1.15.0 sk-video 1.1.10 smach 2.0.1 smach-ros 2.0.1 smclib 1.8.5 sortedcollections 1.1.2 soupsieve 2.0.1 Sphinx 2.4.4 sphinxcontrib-applehelp 1.0.2 sphinxcontrib-devhelp 1.0.2 sphinxcontrib-htmlhelp 1.0.3 sphinxcontrib-jsmath 1.0.1 sphinxcontrib-qthelp 1.0.3 sphinxcontrib-serializinghtml 1.1.4 sphinxcontrib-websupport 1.2.0 spyder 4.0.1 spyder-kernels 1.8.1 SQLAlchemy 1.3.18 sqlalchemy-rqlite HEAD sqlite-web 0.3.6 srt 3.0.0 statsmodels 0.11.1 stompest 2.3.0 tblib 1.6.0 tensorboard 2.1.1 tensorflow-cpu 2.1.0 tensorflow-estimator 2.1.0 termcolor 1.1.0 terminado 0.8.3 testpath 0.4.4 tf 1.12.1 tf-conversions 1.12.1 tf2-geometry-msgs 0.6.5 tf2-kdl 0.6.5 tf2-py 0.6.5 tf2-ros 0.6.5 Theano 1.0.4 topic-tools 1.14.6 tornado 6.0.4 tqdm 4.47.0 traitlets 4.3.3 Twisted 19.10.0 typed-ast 1.4.1 ujson 1.35 urdfdom-py 0.4.3 uritemplate 3.0.1 urllib3 1.25.9 watchdog 0.10.2 wcwidth 0.2.5 webencodings 0.5.1 websocket-client 0.57.0 Werkzeug 1.0.0 wheel 0.34.2 whichcraft 0.6.1 widgetsnbextension 3.5.1 wrapt 1.12.1 wurlitzer 2.0.0 wxPython 4.0.4 xacro 1.13.5 xarray 0.15.0 XlsxWriter 1.2.8 xmltodict 0.12.0 yapf 0.29.0 zict 2.0.0 zipp 3.1.0 zope.interface 4.7.1 ```

int19h commented 4 years ago

Can you try setting PYDEVD_USE_FRAME_EVAL=NO in environment variables, and see if that helps? Since you're launching the program using the debugger, you should be able to set it via "env" in launch.json.

dddvision commented 4 years ago

Setting "env":{"PYDEVD_USE_FRAME_EVAL": "NO"} in launch.json solves the problem. Thank you!

Should I add this to all of my launch.json files until a bug fix is pushed out?

int19h commented 4 years ago

It would probably be easiest to just set it as a global environment variable - that way it'll also apply to "attach" sessions.

dddvision commented 4 years ago

I'm looking forward to a more permanent bug fix.

int19h commented 4 years ago

@fabioz, can you take a look? It might be a rare corner case, but changing behavior of running code like that is very concerning.

fabioz commented 4 years ago

Sure, I'll give priority to this issue.

int19h commented 4 years ago

The crucial requirement for the repro is Python 3.7, although I suspect this has more to do with us only shipping Cython binaries for that.

fabioz commented 4 years ago

Note: I'm able to reproduce this and I'm investigating it (took me a bit more than I expected today because I ended up having to reinstall my Linux VirtualBox image which became corrupted).

fabioz commented 4 years ago

The issue is related to the bytecode modification in the frame eval mode -- I'm still working on it.

fabioz commented 4 years ago

Note: just to give more feedback, I'm still working on this.

I've actually found more issues related to that on the frame eval mode as I started doing some more tests on this, so, I'm actually scrapping the code related to the bytecode rewrite and doing the bytecode rewrite using https://github.com/vstinner/bytecode, which should allow for a better API for bytecode manipulation.

I think I'll be able to finish it next week.

marknelson commented 4 years ago

I'm seeing the same problem, python 3.7, the environment variable fix resolves the problem.

BraveDistribution commented 1 year ago

This is still valid on build: 1.74.3

"PYDEVD_USE_FRAME_EVAL": "NO" doesn't work as a fix anymore.

int19h commented 1 year ago

Are you doing an attach, by chance? Putting stuff in "env" only works for launch, otherwise environment variable has to be set specifically for the process you're attaching to when it's started.

If that doesn't solve the problem, can you please file a new issue?

microsoft / debugpy