Strange issue with debugging Python code

[WildCard65]

OK, so I came across a strange error when attempting to debug an issue with test_httpservers's test_get method (for a deadlock, reference: https://bugs.python.org/issue25095 ) and I came across this error when attaching the Python debugger to a running python_d.exe (Python 3.6.4 built from source cloned from github)

Error:

* ob
object  : <refcnt 0 at 05428A90>
type    : bytes
refcount: 0
address : 05428A90
* op->_ob_prev->_ob_next
NULL
* op->_ob_next->_ob_prev
object  : <refcnt 0 at 05428A90>
type    : bytes
refcount: 0
address : 05428A90
Fatal Python error: UNREF invalid object

Thread 0x00003964 (most recent call first):
  File "C:\Program Files (x86)\Microsoft Visual Studio\2017\Enterprise\Common7\IDE\Extensions\Microsoft\Python\Core\ptvsd\ipcjson.py", line 89 in _buffered_read_line_as_ascii
  File "C:\Program Files (x86)\Microsoft Visual Studio\2017\Enterprise\Common7\IDE\Extensions\Microsoft\Python\Core\ptvsd\ipcjson.py", line 125 in _wait_for_message
  File "C:\Program Files (x86)\Microsoft Visual Studio\2017\Enterprise\Common7\IDE\Extensions\Microsoft\Python\Core\ptvsd\ipcjson.py", line 241 in process_one_message
  File "C:\Program Files (x86)\Microsoft Visual Studio\2017\Enterprise\Common7\IDE\Extensions\Microsoft\Python\Core\ptvsd\debugger.py", line 1687 in loop
  File "C:\Program Files (x86)\Microsoft Visual Studio\2017\Enterprise\Common7\IDE\Extensions\Microsoft\Python\Core\ptvsd\debugger.py", line 127 in thread_starter

Thread 0x0000428c (most recent call first):

Current thread 0x00002428 (most recent call first):
  File "C:\Users\user\Documents\PythonSource\CPython-3.6\lib\test\libregrtest\main.py", line 519 in _main
  File "C:\Users\user\Documents\PythonSource\CPython-3.6\lib\test\libregrtest\main.py", line 509 in main
  File "C:\Users\user\Documents\PythonSource\CPython-3.6\lib\test\libregrtest\main.py", line 584 in main
  File "C:\Users\user\Documents\PythonSource\CPython-3.6\lib\test\__main__.py", line 2 in <module>
  File "C:\Users\user\Documents\PythonSource\CPython-3.6\lib\runpy.py", line 85 in _run_code
  File "C:\Users\user\Documents\PythonSource\CPython-3.6\lib\runpy.py", line 193 in _run_module_as_main

Command line: win32\python_d.exe -m test -v --header -s test_httpservers -u network --wait

Build command line: .\build.bat -e -m -k -t Build -c Debug -p Win32 | Tee-Object -FilePath "..\build.log"

Environment: Visual Studio 2017 (For Python Debugger) Visual Studio 2015 (For Python 3.6 compilation) MSBuild 14.0 (Detected by Python's find_msbuild.bat)

[huguesv]

Thanks for the details.

line 89 in _buffered_read_line_as_ascii is:

temp = self.__socket.recv(1024)

from https://github.com/Microsoft/PTVS/blob/15.6p2/Python/Product/PythonTools/ptvsd/ipcjson.py#L89

I have no idea.

[zooba]

We don't normally test the attach code against debug builds and we make some assumptions that may not be true (particularly where we force the GIL to be created and then spin-loop until we can acquire it without Python knowing).

However, this may also be invalid refcounting on our part that we normally get away with. We probably have to audit our PyDebugAttach C code, but at the same time I'm not sure this is a high enough priority for us to dedicate time to it (and auditing this code for refcounting is going to be laborious and not much fun).

In any case, your original scenario would be better off using mixed-mode debugging (when attaching, manually select both "Python" and "Native" engines). That won't be affected by this issue as it uses completely different code.

[WildCard65]

I can attest that Mixed Mode doesn't produce the crash, but it doesn't mean it doesn't have it's own problems from my usage (seperate from this issue entirely).

[huguesv]

We have a new debug engine in the 15.8 release, if you still see this problem, please open an issue at https://github.com/Microsoft/ptvsd