Timed out while attaching to running Jupyter interactive window or Jupyter notebook

guileryu01 commented 2 years ago

Environment data

debugpy version: 1.6.0
OS and version: Windows 10 Home, OS build 19044.1826
Python version : 3.9.13 Anaconda
Using VS Code or Visual Studio: VS Code version 1.69.2

Actual behavior

Timed out whiling waiting for debugger to attach to running Jupyter interactive window or Jupyter notebook.

Attaching to a regular python shell works for me.

Expected behavior

Be able to attach to these running processes.

Steps to reproduce:

Launch a python Jupyter interactive window and choose Python: Attach using Process Id.

I remember this used to work for me before recent updates to vs code. I've also tried setting logToFile to true in my launch.json, which generates the following log in debugpy.adapter:

I+00000.078: Windows-10-10.0.19044-SP0 AMD64
             CPython 3.9.13 (64-bit)
             debugpy 1.6.2

I+00000.078: debugpy.adapter startup environment:

             System paths:
                 sys.prefix: C:\tools\miniconda3\envs\python39
                 sys.base_prefix: C:\tools\miniconda3\envs\python39
                 sys.real_prefix: <missing>
                 site.getsitepackages(): C:\tools\miniconda3\envs\python39
                                         C:\tools\miniconda3\envs\python39\lib\site-packages(C:\tools\miniconda3\envs\python39\Lib\site-packages)
                 site.getusersitepackages(): C:\Users\XXXX\AppData\Roaming\Python\Python39\site-packages
                 sys.path (site-packages): C:\tools\miniconda3\envs\python39\lib\site-packages(C:\tools\miniconda3\envs\python39\Lib\site-packages)
                 sysconfig.get_path('stdlib'): C:\tools\miniconda3\envs\python39\Lib
                 sysconfig.get_path('platstdlib'): C:\tools\miniconda3\envs\python39\Lib
                 sysconfig.get_path('purelib'): C:\tools\miniconda3\envs\python39\Lib\site-packages
                 sysconfig.get_path('platlib'): C:\tools\miniconda3\envs\python39\Lib\site-packages
                 sysconfig.get_path('include'): C:\tools\miniconda3\envs\python39\Include(C:\tools\miniconda3\envs\python39\include)
                 sysconfig.get_path('scripts'): C:\tools\miniconda3\envs\python39\Scripts
                 sysconfig.get_path('data'): C:\tools\miniconda3\envs\python39
                 os.__file__: C:\tools\miniconda3\envs\python39\lib\os.py(C:\tools\miniconda3\envs\python39\Lib\os.py)
                 threading.__file__: C:\tools\miniconda3\envs\python39\lib\threading.py(C:\tools\miniconda3\envs\python39\Lib\threading.py)

I+00000.094: Listening for incoming Client connections on 127.0.0.1:60189...

I+00000.094: Connecting to client over stdio...

D+00000.094: Starting message loop for channel Client[1]

D+00000.094: Client[1] <-- {
                 "seq": 1,
                 "type": "event",
                 "event": "output",
                 "body": {
                     "category": "telemetry",
                     "output": "ptvsd",
                     "data": {
                         "packageVersion": "1.6.2"
                     }
                 }
             }

D+00000.094: Client[1] --> {
                 "seq": 1,
                 "type": "request",
                 "command": "initialize",
                 "arguments": {
                     "clientID": "vscode",
                     "clientName": "Visual Studio Code",
                     "adapterID": "python",
                     "pathFormat": "path",
                     "linesStartAt1": true,
                     "columnsStartAt1": true,
                     "supportsVariableType": true,
                     "supportsVariablePaging": true,
                     "supportsRunInTerminalRequest": true,
                     "locale": "en-us",
                     "supportsProgressReporting": true,
                     "supportsInvalidatedEvent": true,
                     "supportsMemoryReferences": true
                 }
             }

D+00000.094: Client[1] <-- {
                 "seq": 2,
                 "type": "event",
                 "event": "output",
                 "body": {
                     "category": "telemetry",
                     "output": "debugpy",
                     "data": {
                         "packageVersion": "1.6.2"
                     }
                 }
             }

D+00000.094: /handling #1 request "initialize" from Client[1]/
             Capabilities: {
                 "supportsVariableType": true,
                 "supportsVariablePaging": true,
                 "supportsRunInTerminalRequest": true,
                 "supportsMemoryReferences": true
             }

D+00000.094: /handling #1 request "initialize" from Client[1]/
             Expectations: {
                 "locale": "en-us",
                 "linesStartAt1": true,
                 "columnsStartAt1": true,
                 "pathFormat": "path"
             }

D+00000.094: /handling #1 request "initialize" from Client[1]/
             Client[1] <-- {
                 "seq": 3,
                 "type": "response",
                 "request_seq": 1,
                 "success": true,
                 "command": "initialize",
                 "body": {
                     "supportsCompletionsRequest": true,
                     "supportsConditionalBreakpoints": true,
                     "supportsConfigurationDoneRequest": true,
                     "supportsDebuggerProperties": true,
                     "supportsDelayedStackTraceLoading": true,
                     "supportsEvaluateForHovers": true,
                     "supportsExceptionInfoRequest": true,
                     "supportsExceptionOptions": true,
                     "supportsFunctionBreakpoints": true,
                     "supportsHitConditionalBreakpoints": true,
                     "supportsLogPoints": true,
                     "supportsModulesRequest": true,
                     "supportsSetExpression": true,
                     "supportsSetVariable": true,
                     "supportsValueFormattingOptions": true,
                     "supportsTerminateDebuggee": true,
                     "supportsGotoTargetsRequest": true,
                     "supportsClipboardContext": true,
                     "exceptionBreakpointFilters": [
                         {
                             "filter": "raised",
                             "label": "Raised Exceptions",
                             "default": false,
                             "description": "Break whenever any exception is raised."
                         },
                         {
                             "filter": "uncaught",
                             "label": "Uncaught Exceptions",
                             "default": true,
                             "description": "Break when the process is exiting due to unhandled exception."
                         },
                         {
                             "filter": "userUnhandled",
                             "label": "User Uncaught Exceptions",
                             "default": false,
                             "description": "Break when exception escapes into library code."
                         }
                     ],
                     "supportsStepInTargetsRequest": true
                 }
             }

D+00000.109: Client[1] --> {
                 "seq": 2,
                 "type": "request",
                 "command": "attach",
                 "arguments": {
                     "name": "Python: Attach using Process Id",
                     "type": "python",
                     "request": "attach",
                     "processId": "3536",
                     "logToFile": true,
                     "justMyCode": false,
                     "__configurationTarget": 5,
                     "debugOptions": [
                         "DebugStdLib",
                         "RedirectOutput",
                         "FixFilePathCase",
                         "WindowsClient",
                         "ShowReturnValue"
                     ],
                     "host": "localhost",
                     "showReturnValue": true,
                     "workspaceFolder": "d:\\projects\\sandbox",
                     "pathMappings": [
                         {
                             "localRoot": "d:\\projects\\sandbox",
                             "remoteRoot": "d:\\projects\\sandbox"
                         }
                     ],
                     "__sessionId": "003c8a3f-c4e3-41d6-b7e8-a54f0dcf637c"
                 }
             }

I+00000.109: /handling #2 request "attach" from Client[1]/
             Listening for incoming Server connections on 127.0.0.1:60190...

I+00000.109: /handling #2 request "attach" from Client[1]/
             Spawning attach-to-PID debugger injector: ['C:\\tools\\miniconda3\\envs\\python39\\python.exe', 'c:\\Users\\XXXX\\.vscode\\extensions\\ms-python.python-2022.10.1\\pythonFiles\\lib\\python\\debugpy\\adapter/../..\\debugpy', '--connect', '127.0.0.1:60190', '--adapter-access-token', '747b7ebd610657e96abc61673cedce6a38a4e6fec9172073f1835c404ca07edc', '--pid', '3536']

D+00000.109: /handling #2 request "attach" from Client[1]/
             Client[1] <-- {
                 "seq": 4,
                 "type": "event",
                 "event": "debugpyWaitingForServer",
                 "body": {
                     "host": "127.0.0.1",
                     "port": 60190
                 }
             }

I+00000.109: /handling #2 request "attach" from Client[1]/
             Session[1] waiting for connection from debug server...

I+00000.922: Injector[PID=3536] output:
             b'Running executable to inject dll.'

I+00000.922: Injector[PID=3536] output:
             b'OpenProcess with pid: 3536'

I+00000.922: Injector[PID=3536] output:
             b'VirtualAllocEx in pid: 3536'

I+00000.922: Injector[PID=3536] output:
             b'WriteProcessMemory in pid: 3536'

I+00000.922: Injector[PID=3536] output:
             b'loadLibraryAddress: 3536'

I+00000.922: Injector[PID=3536] output:
             b'Waiting for LoadLibraryA to complete.'

I+00000.922: Injector[PID=3536] output:
             b'Ok, finished dll injection.'

I+00000.938: Injector[PID=3536] output:
             b'Running executable to inject dll.'

I+00000.938: Injector[PID=3536] output:
             b'OpenProcess with pid: 3536'

I+00000.938: Injector[PID=3536] output:
             b'VirtualAllocEx in pid: 3536'

I+00000.938: Injector[PID=3536] output:
             b'WriteProcessMemory in pid: 3536'

I+00000.938: Injector[PID=3536] output:
             b'loadLibraryAddress: 3536'

I+00000.938: Injector[PID=3536] output:
             b'Waiting for LoadLibraryA to complete.'

I+00000.938: Injector[PID=3536] output:
             b'Ok, finished dll injection.'

I+00005.953: Injector[PID=3536] output:
             b'--- Connecting to 64 bits target (current process is: 64) ---'

I+00005.953: Injector[PID=3536] output:
             b''

I+00005.953: Injector[PID=3536] output:
             b'--- Injecting attach dll: attach_amd64.dll into pid: 3536 ---'

I+00005.953: Injector[PID=3536] output:
             b''

I+00005.953: Injector[PID=3536] output:
             b'--- Injecting run code dll: run_code_on_dllmain_amd64.dll into pid: 3536 ---'

I+00005.953: Injector[PID=3536] output:
             b'--- Finished dll injection ---'

I+00005.953: Injector[PID=3536] output:
             b''

I+00006.047: Injector[PID=3536] exited.

I+00015.109: All debug servers disconnected; waiting for remaining sessions...

E+00015.125: /handling #2 request "attach" from Client[1]/
             Traceback (most recent call last):
               File "c:\Users\XXXX\.vscode\extensions\ms-python.python-2022.10.1\pythonFiles\lib\python\debugpy\adapter/../..\debugpy\common\messaging.py", line 1014, in __init__
                 raise self
             debugpy.common.messaging.MessageHandlingError: Timed out waiting for debug server to connect.

             Stack where logged:
               File "C:\tools\miniconda3\envs\python39\lib\threading.py", line 937, in _bootstrap
                 self._bootstrap_inner()
               File "C:\tools\miniconda3\envs\python39\lib\threading.py", line 980, in _bootstrap_inner
                 self.run()
               File "C:\tools\miniconda3\envs\python39\lib\threading.py", line 917, in run
                 self._target(*self._args, **self._kwargs)
               File "c:\Users\XXXX\.vscode\extensions\ms-python.python-2022.10.1\pythonFiles\lib\python\debugpy\adapter/../..\debugpy\common\messaging.py", line 1458, in _run_handlers
                 handler()
               File "c:\Users\XXXX\.vscode\extensions\ms-python.python-2022.10.1\pythonFiles\lib\python\debugpy\adapter/../..\debugpy\common\messaging.py", line 700, in _handle
                 result = handler(self)
               File "c:\Users\XXXX\.vscode\extensions\ms-python.python-2022.10.1\pythonFiles\lib\python\debugpy\adapter/../..\debugpy\adapter\components.py", line 91, in lock_and_handle
                 return f(self, message)
               File "c:\Users\XXXX\.vscode\extensions\ms-python.python-2022.10.1\pythonFiles\lib\python\debugpy\adapter/../..\debugpy\adapter\clients.py", line 210, in handle
                 f(self, request)
               File "c:\Users\XXXX\.vscode\extensions\ms-python.python-2022.10.1\pythonFiles\lib\python\debugpy\adapter/../..\debugpy\adapter\clients.py", line 525, in attach_request
                 raise request.cant_handle(
               File "c:\Users\XXXX\.vscode\extensions\ms-python.python-2022.10.1\pythonFiles\lib\python\debugpy\adapter/../..\debugpy\common\messaging.py", line 524, in cant_handle
                 return self.error(MessageHandlingError, *args, **kwargs)
               File "c:\Users\XXXX\.vscode\extensions\ms-python.python-2022.10.1\pythonFiles\lib\python\debugpy\adapter/../..\debugpy\common\messaging.py", line 512, in error
                 exc = exc_type(reason, self, silent)  # will log it
               File "c:\Users\XXXX\.vscode\extensions\ms-python.python-2022.10.1\pythonFiles\lib\python\debugpy\adapter/../..\debugpy\common\messaging.py", line 1016, in __init__
                 log.swallow_exception()
               File "c:\Users\XXXX\.vscode\extensions\ms-python.python-2022.10.1\pythonFiles\lib\python\debugpy\adapter/../..\debugpy\common\log.py", line 215, in swallow_exception
                 _exception(format_string, *args, **kwargs)

D+00015.125: /handling #2 request "attach" from Client[1]/
             Client[1] <-- {
                 "seq": 5,
                 "type": "response",
                 "request_seq": 2,
                 "success": false,
                 "command": "attach",
                 "message": "Timed out waiting for debug server to connect."
             }

E+00015.125: /handling #2 request "attach" from Client[1]/
             Handler 'Client._start_message_handler.<locals>.handle' (file 'c:\\Users\\XXXX\\.vscode\\extensions\\ms-python.python-2022.10.1\\pythonFiles\\lib\\python\\debugpy\\adapter/../..\\debugpy\\adapter\\components.py', line 194)
             couldn't handle #2 request "attach" from Client[1]:
             Timed out waiting for debug server to connect.

D+00015.156: Client[1] --> {
                 "seq": 3,
                 "type": "request",
                 "command": "disconnect",
                 "arguments": {
                     "restart": false
                 }
             }

I+00015.156: /handling #3 request "disconnect" from Client[1]/
             client requested "disconnect"; finalizing Session[1].

D+00015.156: /handling #3 request "disconnect" from Client[1]/
             Client[1] <-- {
                 "seq": 6,
                 "type": "event",
                 "event": "terminated"
             }

I+00015.156: /handling #3 request "disconnect" from Client[1]/
             Session[1] finalized.

D+00015.156: /handling #3 request "disconnect" from Client[1]/
             Client[1] <-- {
                 "seq": 7,
                 "type": "response",
                 "request_seq": 3,
                 "success": true,
                 "command": "disconnect"
             }

int19h commented 2 years ago

Does it work if you use debugpy.listen() etc instead?

guileryu01 commented 2 years ago

Yes. I tried adding

import debugpy
debugpy.listen(("localhost", 5678))
debugpy.wait_for_client()

in the code, and launching the debugger with the following config:

{
    "name": "Python: Attach",
    "type": "python",
    "request": "attach",
    "connect": {
      "host": "localhost",
      "port": 5678
    }
}

This way, the debugger was able to attach to a Jupyter python session.

int19h commented 2 years ago

I'm not entirely sure what goes wrong here, but I'm also not quite surprised because Jupyter has its own integrated debugging facilities, so our and their might be clashing here. Attach-by-PID is particularly fragile in that regard due to all the hacks necessary to hijack an already-running interpreter.

Have you considered using the built-in Jupyter debugging support in VSCode? Aside from taking care of all this for you, it will also do things such as setting breakpoints in code cells and stepping through them, which doesn't work well in a regular attach debug session.

https://code.visualstudio.com/docs/datascience/jupyter-notebooks#_debug-a-jupyter-notebook

guileryu01 commented 2 years ago

Thanks for the suggestion. Yes both "Debug cell" or "Run by line" work, which is also what I'm using. However I didn't find an equivalent function for Jupyter integrative window, and attaching by process ID is the only way that I found that worked until recently.

rchiodo commented 2 years ago

Interactive window should have its own 'Debug Cell' (you may be not seeing it in your python script if you're not putting the cell markers # %%)

https://code.visualstudio.com/docs/python/jupyter-support-py#_jupyter-code-cells

guileryu01 commented 2 years ago

I see, although it would still be nice if I could leave the modules alone and do all the testing in the python interactive window. Is it possible to trigger this debug function from python interactive directly, e.g. same as what Ctrl+Shift+Alt+Enter does in an actual Jupyter Notebook?

rchiodo commented 2 years ago

There's no way to 'debug' the current selection, but you can debug the entire python file with 'Jupyter: Debug current file in Interactive Window'.

Debug current cell requires the cell markers be available, but the Debug current file will just load the entire python file as a single cell and let you step over it one line at a time.

dokutoshi commented 1 year ago

There's no way to 'debug' the current selection, but you can debug the entire python file with 'Jupyter: Debug current file in Interactive Window'.

Debug current cell requires the cell markers be available, but the Debug current file will just load the entire python file as a single cell and let you step over it one line at a time.

Any reason that the Debug Current File in Interactive Window or Jupyter: Run Current File in Python Interactive Window disappears as an option in Jupyter notebook?

rchiodo commented 1 year ago

Any reason that the Debug Current File in Interactive Window or Jupyter: Run Current File in Python Interactive Window disappears as an option in Jupyter notebook?

Notebooks have their own way to debug cells. You can debug from the UI on the notebook itself.

https://code.visualstudio.com/docs/datascience/jupyter-notebooks#_debug-a-jupyter-notebook

microsoft / debugpy