[FeatureRequest] Remote Debugging Capabilities for Azure Functions

[cmcconomyfwig]

tldr: I am looking for a way to successfully remotely debug a python function running in an azure function app instance

---

I wish to be able to connect VSCode's remote debugger to a running python instance on Azure.

I got as far as pip install debugpy and, ssh'ing into my azure environment from /home/site/wwwroot:
PATH=$PATH:/usr/share PYTHONPATH=./.python_packages/lib/site-packages/ python3 -m debugpy --listen localhost:9000 --pid <pid> where pid is from ls -lrt | grep python.

I get the following error:

PYDEVD_GDB_SCAN_SHARED_LIBRARIES not set (scanning all libraries for needed symbols).
Running: gdb --nw --nh --nx --pid 111 --batch --eval-command='set scheduler-locking off' --eval-command='set architecture auto' --eval-command='call (void*)dlopen("/home/site/wwwroot/.python_packages/lib/site-packages/debugpy/_vendored/pydevd/pydevd_attach_to_process/attach_linux_amd64.so", 2)' --eval-command='sharedlibrary attach_linux_amd64' --eval-command='call (int)DoAttach(0, "import codecs;import json;import sys;decode = lambda s: codecs.utf_8_decode(bytearray(s))[0] if s is not None else None;script_dir = decode([47, 104, 111, 109, 101, 47, 115, 105, 116, 101, 47, 119, 119, 119, 114, 111, 111, 116, 47, 46, 112, 121, 116, 104, 111, 110, 95, 112, 97, 99, 107, 97, 103, 101, 115, 47, 108, 105, 98, 47, 115, 105, 116, 101, 45, 112, 97, 99, 107, 97, 103, 101, 115, 47, 100, 101, 98, 117, 103, 112, 121, 47, 115, 101, 114, 118, 101, 114]);setup = json.loads(decode([123, 34, 109, 111, 100, 101, 34, 58, 32, 34, 108, 105, 115, 116, 101, 110, 34, 44, 32, 34, 97, 100, 100, 114, 101, 115, 115, 34, 58, 32, 91, 34, 108, 111, 99, 97, 108, 104, 111, 115, 116, 34, 44, 32, 57, 48, 48, 48, 93, 44, 32, 34, 119, 97, 105, 116, 95, 102, 111, 114, 95, 99, 108, 105, 101, 110, 116, 34, 58, 32, 102, 97, 108, 115, 101, 44, 32, 34, 108, 111, 103, 95, 116, 111, 34, 58, 32, 110, 117, 108, 108, 44, 32, 34, 97, 100, 97, 112, 116, 101, 114, 95, 97, 99, 99, 101, 115, 115, 95, 116, 111, 107, 101, 110, 34, 58, 32, 110, 117, 108, 108, 125]));sys.path.insert(0, script_dir);import attach_pid_injected;del sys.path[0];attach_pid_injected.attach(setup);", 0)'
/bin/sh: 1: gdb: Permission denied
E+00000.144: Code injection into PID=111 failed:

             Traceback (most recent call last):
               File "/home/site/wwwroot/.python_packages/lib/site-packages/debugpy/server/cli.py", line 391, in attach_to_pid
                 add_code_to_python_process.run_python_code(
               File "/home/site/wwwroot/.python_packages/lib/site-packages/debugpy/_vendored/pydevd/pydevd_attach_to_process/add_code_to_python_process.py", line 471, in run_python_code_linux
                 subprocess.check_call(' '.join(cmd), shell=True, env=env)
               File "/usr/local/lib/python3.9/subprocess.py", line 373, in check_call
                 raise CalledProcessError(retcode, cmd)
             subprocess.CalledProcessError: Command 'gdb --nw --nh --nx --pid 111 --batch --eval-command='set scheduler-locking off' --eval-command='set architecture auto' --eval-command='call (void*)dlopen("/home/site/wwwroot/.python_packages/lib/site-packages/debugpy/_vendored/pydevd/pydevd_attach_to_process/attach_linux_amd64.so", 2)' --eval-command='sharedlibrary attach_linux_amd64' --eval-command='call (int)DoAttach(0, "import codecs;import json;import sys;decode = lambda s: codecs.utf_8_decode(bytearray(s))[0] if s is not None else None;script_dir = decode([47, 104, 111, 109, 101, 47, 115, 105, 116, 101, 47, 119, 119, 119, 114, 111, 111, 116, 47, 46, 112, 121, 116, 104, 111, 110, 95, 112, 97, 99, 107, 97, 103, 101, 115, 47, 108, 105, 98, 47, 115, 105, 116, 101, 45, 112, 97, 99, 107, 97, 103, 101, 115, 47, 100, 101, 98, 117, 103, 112, 121, 47, 115, 101, 114, 118, 101, 114]);setup = json.loads(decode([123, 34, 109, 111, 100, 101, 34, 58, 32, 34, 108, 105, 115, 116, 101, 110, 34, 44, 32, 34, 97, 100, 100, 114, 101, 115, 115, 34, 58, 32, 91, 34, 108, 111, 99, 97, 108, 104, 111, 115, 116, 34, 44, 32, 57, 48, 48, 48, 93, 44, 32, 34, 119, 97, 105, 116, 95, 102, 111, 114, 95, 99, 108, 105, 101, 110, 116, 34, 58, 32, 102, 97, 108, 115, 101, 44, 32, 34, 108, 111, 103, 95, 116, 111, 34, 58, 32, 110, 117, 108, 108, 44, 32, 34, 97, 100, 97, 112, 116, 101, 114, 95, 97, 99, 99, 101, 115, 115, 95, 116, 111, 107, 101, 110, 34, 58, 32, 110, 117, 108, 108, 125]));sys.path.insert(0, script_dir);import attach_pid_injected;del sys.path[0];attach_pid_injected.attach(setup);", 0)'' returned non-zero exit status 127.

             Stack where logged:
               File "/usr/local/lib/python3.9/runpy.py", line 197, in _run_module_as_main
                 return _run_code(code, main_globals, None,
               File "/usr/local/lib/python3.9/runpy.py", line 87, in _run_code
                 exec(code, run_globals)
               File "/home/site/wwwroot/.python_packages/lib/site-packages/debugpy/__main__.py", line 39, in <module>
                 cli.main()
               File "/home/site/wwwroot/.python_packages/lib/site-packages/debugpy/server/cli.py", line 430, in main
                 run()
               File "/home/site/wwwroot/.python_packages/lib/site-packages/debugpy/server/cli.py", line 398, in attach_to_pid
                 log.reraise_exception("Code injection into PID={0} failed:", pid)
               File "/home/site/wwwroot/.python_packages/lib/site-packages/debugpy/common/log.py", line 222, in reraise_exception
                 _exception(format_string, *args, **kwargs)

Traceback (most recent call last):
  File "/usr/local/lib/python3.9/runpy.py", line 197, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/usr/local/lib/python3.9/runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "/home/site/wwwroot/.python_packages/lib/site-packages/debugpy/__main__.py", line 39, in <module>
    cli.main()
  File "/home/site/wwwroot/.python_packages/lib/site-packages/debugpy/server/cli.py", line 430, in main
    run()
  File "/home/site/wwwroot/.python_packages/lib/site-packages/debugpy/server/cli.py", line 398, in attach_to_pid
    log.reraise_exception("Code injection into PID={0} failed:", pid)
  File "/home/site/wwwroot/.python_packages/lib/site-packages/debugpy/server/cli.py", line 391, in attach_to_pid
    add_code_to_python_process.run_python_code(
  File "/home/site/wwwroot/.python_packages/lib/site-packages/debugpy/_vendored/pydevd/pydevd_attach_to_process/add_code_to_python_process.py", line 471, in run_python_code_linux
    subprocess.check_call(' '.join(cmd), shell=True, env=env)
  File "/usr/local/lib/python3.9/subprocess.py", line 373, in check_call
    raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command 'gdb --nw --nh --nx --pid 111 --batch --eval-command='set scheduler-locking off' --eval-command='set architecture auto' --eval-command='call (void*)dlopen("/home/site/wwwroot/.python_packages/lib/site-packages/debugpy/_vendored/pydevd/pydevd_attach_to_process/attach_linux_amd64.so", 2)' --eval-command='sharedlibrary attach_linux_amd64' --eval-command='call (int)DoAttach(0, "import codecs;import json;import sys;decode = lambda s: codecs.utf_8_decode(bytearray(s))[0] if s is not None else None;script_dir = decode([47, 104, 111, 109, 101, 47, 115, 105, 116, 101, 47, 119, 119, 119, 114, 111, 111, 116, 47, 46, 112, 121, 116, 104, 111, 110, 95, 112, 97, 99, 107, 97, 103, 101, 115, 47, 108, 105, 98, 47, 115, 105, 116, 101, 45, 112, 97, 99, 107, 97, 103, 101, 115, 47, 100, 101, 98, 117, 103, 112, 121, 47, 115, 101, 114, 118, 101, 114]);setup = json.loads(decode([123, 34, 109, 111, 100, 101, 34, 58, 32, 34, 108, 105, 115, 116, 101, 110, 34, 44, 32, 34, 97, 100, 100, 114, 101, 115, 115, 34, 58, 32, 91, 34, 108, 111, 99, 97, 108, 104, 111, 115, 116, 34, 44, 32, 57, 48, 48, 48, 93, 44, 32, 34, 119, 97, 105, 116, 95, 102, 111, 114, 95, 99, 108, 105, 101, 110, 116, 34, 58, 32, 102, 97, 108, 115, 101, 44, 32, 34, 108, 111, 103, 95, 116, 111, 34, 58, 32, 110, 117, 108, 108, 44, 32, 34, 97, 100, 97, 112, 116, 101, 114, 95, 97, 99, 99, 101, 115, 115, 95, 116, 111, 107, 101, 110, 34, 58, 32, 110, 117, 108, 108, 125]));sys.path.insert(0, script_dir);import attach_pid_injected;del sys.path[0];attach_pid_injected.attach(setup);", 0)'' returned non-zero exit status 127.

I do think the following line from the above dump is key: /bin/sh: 1: gdb: Permission denied

---

Locally, on my mac, I can connect using debugpy's debugpy.listen(("0.0.0.0", port)) -- but from my limited understanding, the remote isn't working because gdb either isn't distributed in the provided python container, OR user permissions aren't right (although when I ssh into the server I am root!) -- or, perhaps the container is missing certain permission grants?

Can you let me know how I can remote-debug a python app running inside Azure?

[bhagyshricompany]

Thanks for Feedback will inform you on this asap.

[cmcconomyfwig]

Thanks for Feedback will inform you on this asap.

I've also attempted the following:

1. run az webapp create-remote-connection <...> to create an ssh tunnel to the function
2. open vscode
3. using the Remote-SSH plugin, connect to host root@127.0.0.1:<port>
4. enter password Docker! as provided by step 1 above

In this case, VSCode goes into a reconnection loop while trying to establish its vscode server footprint. I tried toggling the vscode setting "Use FLock" as I've seen advised in some places, but it didn't change anything.

[bhagyshricompany]

@cmcconomyfwig you can refer this document.for feature will do in future and update . https://code.visualstudio.com/docs/editor/debugging#_remote-debugging

[cmcconomyfwig]

@cmcconomyfwig you can refer this document.for feature will do in future and update . https://code.visualstudio.com/docs/editor/debugging#_remote-debugging

Have you successfully connected to a cloud python function instance following these directions? I'd like to figure out how to get the debug session going.

[gavin-aguiar]

@cmcconomyfwig Thanks for reporting this feature request. We currently do not support remote debugging. We have added this feature request to our backlog and will update this github issue with any progress.

[cmcconomyfwig]

@cmcconomyfwig Thanks for reporting this feature request. We currently do not support remote debugging. We have added this feature request to our backlog and will update this github issue with any progress.

Thank you.. i believe the most direct way to support this would be to troubleshoot the vscode remote ssh capability

[cmcconomyfwig]

Update for anyone who visits;

It appears that sometimes, the VSCode Remote-SSH extensions works when operating through the localhost ssh session you can establish as above via az webapp create-remote-connection !

After this point, I had to:

1. browse to a remote folder location with write access (so that vscode can write to ./.vscode - this means you can't mount directly into a read only file system such as that created by a zip-deploy)
2. from the remote terminal, apt install pdb which is required by vscode python
3. install the ms-python.python plugin on the remote vscode server
4. in vscode, run -> debug -> attach to process -> yourpid
5. add debug point in your code under /home/site/wwwroot/

Hope this helps anyone else looking for this.