VSCode debugging recipe (python)

[AreWeDreaming]

Has anyone tried to debug a muscle3 application with VSCode? So far I have always just debugged the macro/micro separately, and then relied on logs for integration tests. I tried the following:

{
        "name": "MUSCLE3 ecrad test",
        "type": "python",
        "request": "launch",
        "program": "/fusion/projects/codes/conda/omega/dev_envs/denks/IMAS_env/bin/muscle_manager",
        "console": "integratedTerminal",
        "justMyCode": false,
        "cwd": "/home/denks/git/ECRad_core/muscle3_test/",
        "args": [
            "--start-all",
            "configuration.ymmsl"
            ]
}

But I cannot launch this due to issues with the arguments. I am aware of this but I don't even know how to launch the muscle_manager so that vscode can attach.

Obviously, this is a nice to have. If none of the dev team is working with VSCode it is probably not worth the effort.

[LourensVeen]

I don't use VSCode myself, so I haven't tried this, but VSCode is used commonly enough that it could be worthwhile adding some documentation at least.

The thing with debugging the manager however is that it doesn't actually do very much. If there's an issue with resource planning, starting the instances, or connecting them, then debugging the manager will help, but otherwise it just sits there. Unless VSCode can jump into Fortran code started as a subprocess from a Python program it started? Then you could debug the whole thing I guess, at least when running locally.

So two questions:

1) Which kinds of issues would you like to debug? 2) Could you post the error message regarding the arguments you're getting, if any?

[AreWeDreaming]

So I managed to start the debugger for the muscle_manager with the following debug config:

"name": "MUSCLE3",
        "type": "python",
        "request": "launch",
        "program": "<path to muscle_manager>",
        "console": "integratedTerminal",
        "justMyCode": false,
        "cwd": "<my cwd>",
        "args": [
            "--start-all",
            "configuration.ymmsl"
            ]
        }

However, this does not allow me to debug the python macro application that the muscle_manager starts. I tried placing breakpoints in the application, but I guess the way the python application is started doesn't allow it. I have used the python debugger in conjunction with the multiprocessing python module in the past and that worked just fine. Is there any way I can debug my macro application that is completely written in python? The micro Fortran application is much more stable, and debugging it is much lower priority, although it would still be nice.

[LourensVeen]

Ah, I see. So the examples in the documentation explain how to debug native applications, but you cannot debug a Python component using gdb. That makes sense. And you could use pdb in the same way as gdb, but it's much less user-friendly than the VSCode interface.

So would it help if you could start your Python code inside VSCode in the same way as the documentation shows how to use a C++ or Fortran code in gdb? I searched around a bit for that and found an issue: https://github.com/microsoft/vscode/issues/10979 and a VSCode extension: https://marketplace.visualstudio.com/items?itemName=fabiospampinato.vscode-debug-launcher

It seems that the latter may let you run a Python program under VSCode from the command line, which you could then do from the yMMSL file, causing VSCode to pop up with the debugger open on your Python script when you start the MUSCLE3 simulation.

As for VSCode not being able to see the launched process, muscle_manager runs QCG-PilotJob in a background process (this is necessary because there's a mix of threads, processes and async and this is the only way to keep the whole thing from crashing), and QCG-PJ then launches your Python program in different ways depending on the environment it runs in. I think that's all just a bit too much for VSCode to follow maybe.

[maarten-ic]

If you want to debug the macro process launched by the muscle_manager, I think it'll be best to try one of the following:

1. Follow the Local script debugging recipe for VSCode.

2. Use debugpy to run the macro process. After launching you can connect the VSCode debugger as in the link of the previous point. You can do this by updating your yaml configuration from

   implementations:
     macro:
       executable: python
       args: macro.py

   to

   implementations:
     macro:
       executable: python
       args: -m debugpy --listen localhost:5678 --wait-for-client macro.py

[AreWeDreaming]

So first of much thanks @LourensVeen and @maarten-ic for your help. I followed @maarten-ic instructions and the linked VSCode instructions and got debugpy to attach. That emboldened me and I wanted to get debugger to attach to both, the Python Macro, and the Fortran Micro.

It took me about half a day to stich everything together buuuut it actually works. Note that this should also work for any C++ application, and should also allow for debugging on remote machines IF the ports are open. Just replace localhost below with the hostname/IP of your target machine, then start muscle_manager on the remote machine.

Here is the recipe:

Requirements:

• VSCode Python plugin
• VSCode C/C++ extension
• gdbserver
• debugpy python module

  Launch configuration in VSCode (launch.json)

  {
  "version": "0.2.0",
  "configurations": [
  {
          "name": "Python attach",
          "type": "python",
          "request": "attach",
          "connect": {
              "host": "localhost",
              "port": 5678
          }
      },
      {
          "name": "Fortran attach",
          "type": "cppdbg",
          "MIMode": "gdb",
          "request": "launch",
          "program": "<path to your application>",
          "miDebuggerServerAddress": "localhost:5679",
          "cwd": "${workspaceRoot}/muscle3"
      }
  ]
  }

  I got the info on how to configure the Fortran attach block from here

Muscle3 configuration (debug_configuration.ymmsl):

  Marco:
    executable: xterm
    args:
      - "-e"
      - "python"
      - "<python entry point>"
      - "<more args if needed>"
  Micro:
    executable: xterm
    args:
    - "-e"
    - "gdbserver"
    - ":5679"
    - "<Fortran entry port>"
    - "<more args if needed>"

debugpy settings in your python entry point script:

You just need to add the following lines:

if __name__ == "__main__":
    import debugpy
    # 5678 is the default attach port in the VS Code debug configurations. Unless a host and port are specified, host defaults to 127.0.0.1
    debugpy.listen(5678)
    print("Waiting for debugger attach")
    debugpy.wait_for_client()
    <your code here>

Feel free to put that into your documentation on readthedocs if you find it useful.

P.S. I tried it without the xterm first but that didn't work.