Rewrite debugger for Godot 4 support + improved maintainability

[DaelonSuzuka]

This is a continuation of https://github.com/godotengine/godot-vscode-plugin/pull/400 by @RedMser. I'm opening this as a draft PR so I have a place to keep some notes.

Current state:

The main thing holding this back, besides some issues with the debugger, is compat with 3.x.

Keeping compatibility with multiple Godot versions will probably double the size of this PR, due to having to create duplicates of every system (debugger, connection, etc.) which must be interchangeable. Not to speak of automatic version detection, which I'm not sure how it should be done just yet.

It may be possible to integrate the changes in #387 into this PR.

[Emi1305]

What's the status of this PR? I'm creating a new project on Godot 4 and this would be really helpful to me

[DaelonSuzuka]

What's the status of this PR?

Not done.

Due to some other PRs from the queue getting merged this week, it's possible that I'll be able to make progress on this soon ™️, but I'm not making any promises.

[DaelonSuzuka]

I have both debugger versions living together now. 3 works in both launch and attach mode, but 4 ONLY works in launch mode. I can't get it to connect to a running editor at all. There seem to be some changes in godot's debugger internals, but its difficult for me to tell the extent of them.

There are a number of other less severe problems with the scene tree or inspector or cleaning up after a session stops, but those should all be fixable with a little effort.

[moonraymurray]

Hello I ended up building this PR and testing it with my project in Godot 4, so far it is working correctly, had to make a few minor changes to my settings and launch to adjust to your modifications. I get debug messages now and also I can pause the code. Also seems as though the variables viewer and breakpoints work as intended. I would review the code but I am not well versed in JS it so I am just leaving a comment. EDIT: I couldn't get the scene preview working but I have never really used that so I don't know what working looks like either. So far this PR ticks all the boxes, if I encounter anything else I will make another comment.

[DaelonSuzuka]

Thanks, @moonraymurray, that's great to hear. Can I assume you were using the debugger in launch mode, ie "launch_game_instance": true? I still haven't gotten attach mode to work ("launch_game_instance": false), and it would be very interesting if it that worked for you.

[moonraymurray]

using "launch_game_instance": true, Yes I was able to use the debugger using that setting. Here is my launch code.

{
    "version": "0.2.0",
    "configurations": [
        {
            "name": "Godot",
            "type": "godot4",
            "request": "launch",
            "project": "${workspaceFolder}",
            "port": 6007,
            "address": "127.0.0.1",
            "launch_game_instance": true,
            "launch_scene": false,
        }
    ]
}

And here is my settings.json, I am using a one or 2 other extensions aswell.

{
    "editor.inlineSuggest.enabled": true,
    "explorer.confirmDelete": false,
    "update.showReleaseNotes": false,
    "launch": {

        "configurations": [],
        "compounds": []
    },
    "theme-by-language.themes": {
        "*": "Default Dark+",
        "gdscript": "GDScript"
    },
    "godotTools.editorPath.godot4": "C:/Users/Neo/Downloads/Godot_v4.0.1-stable_win64.exe//Godot_v4.0.1-stable_win64.exe",
    "godotTools.lsp.serverPort": 6005,
}

I had to delete references to the other editor paths(default or 3) to get it to run. Sorry about the formatting

[DaelonSuzuka]

Made some progress on the Godot 3 debugger. I removed the launch_game_instance: true configuration option because it's the same thing as the request: launch option, which is an actual VSCode feature. After implementing the various API interfaces and splitting the code, it looks like most of the behavior for both session types is correct now. Some issues remain around closing the game/killing the process but the actual connection works now.

My next task is to doing the same thing to the Godot 4 side, and then hopefully these improvements will help me figure out why Godot 4 attach sessions don't work.

I intend to call this PR done once all 4 debugger modes (G3 Launch, G3 Attach, G4 Launch, G4 Attach) are working more-or-less correctly.

[Emi1305]

I'm not able to use the debugger (Compiled from you branch) on windows with the following launch.json file:

{
    // Use IntelliSense to learn about possible attributes.
    // Hover to view descriptions of existing attributes.
    // For more information, visit: https://go.microsoft.com/fwlink/?linkid=830387
    "version": "0.2.0",
    "configurations": [
        {
            "name": "GDScript Godot",
            "type": "godot4",
            "request": "launch",
            "project": "${workspaceFolder}",
            "port": 6007,
            "debugServer": 6006,
            "address": "127.0.0.1",
            "launch_game_instance": true,
            "launch_scene": true
        }
    ]
}

Using wireshark to debug the communication I observe this error message:

{"body":{"error":{"format":"The editor and client are working on different paths; the client is on \"{clientPath}\", but the editor is on \"{editorPath}\"","id":1,"sendTelemetry":false,"showUser":false,"variables":{"clientPath":"C:\\Users\\Emiliano\\Documents\\Sound of Defeat","editorPath":"C:/Users/Emiliano/Documents/Sound of Defeat"}}},"command":"launch","message":"wrong_path","request_seq":2,"seq":3,"success":false,"type":"response"}

The error comes from vscode sending the project directory using \\ to split the directories while godot users /. Hardcoding the path to use / on launch.json is a workaround to get it working

Edit: I just noticed that, while the workaround allows me to launch the game and the external debugger to get attached. Breakpoints are not synced due to similar issue. This is the output I see on wireshark:

{"command":"setBreakpoints","arguments":{"source":{"name":"Player3D.gd","path":"c:\\Users\\Emiliano\\Documents\\Sound of Defeat\\src\\player\\Player3D.gd"},"lines":[],"breakpoints":[],"sourceModified":false},"type":"request","seq":7}

{"body":{"error":{"format":"The editor and client are working on different paths; the client is on \"{clientPath}\", but the editor is on \"{editorPath}\"","id":1,"sendTelemetry":false,"showUser":false,"variables":{"clientPath":"c:\\Users\\Emiliano\\Documents\\Sound of Defeat\\src\\player\\Player3D.gd","editorPath":"C:/Users/Emiliano/Documents/Sound of Defeat"}}},"command":"setBreakpoints","message":"wrong_path","request_seq":7,"seq":497,"success":false,"type":"response"}

[DaelonSuzuka]

@Emi1305 That's excellent information, thank you. I'm planning to get some work done on the godot 4 side of things this week, and with any luck I can get this path stuff sorted out.

It would be great if you could check back and test it again next weekend, maybe.

[clorl]

Would love to see this become a reality in the near future

[seffa1]

This is amazing thank you for the work you've done so far on this !

[DaelonSuzuka]

Quick status update: I've mostly completely a massive internal refactor to simplify the debugger code. Lots of features still don't work or don't work perfectly, but now it should actually be possible for me to trace out what's happening.

• Godot 3
  • launch: mostly working, some issues with closing/stopping game and resetting cleanly
  • attach: working, some issues with resetting cleanly
  • output: working
  • breakpoints: mostly working
  • scene tree: mostly working
  • inspector: mostly working
• Godot 4
  • launch: mostly working,
  • attach: not working
  • output: working
  • breakpoints: game stops, vscode ui does not react
  • scene tree: mostly working
  • inspector: not working

Features I haven't even touched yet:

• stack dumps
• variable inspection
• watch points
• evaluating expressions

I want to get sessions (launch/attach, starting, connecting, stopping, resetting) and breakpoints(pause/resume, stepping, stop) working solidly in both versions, and then I can get some volunteer testers and assess the remaining feature list.

[2fd5]

I want to get sessions (launch/attach, starting, connecting, stopping, resetting) and breakpoints(pause/resume, stepping, stop) working solidly in both versions, and then I can get some volunteer testers and assess the remaining feature list.

Count me in for testing

[ryanabx]

I want to get sessions (launch/attach, starting, connecting, stopping, resetting) and breakpoints(pause/resume, stepping, stop) working solidly in both versions, and then I can get some volunteer testers and assess the remaining feature list.

I'd also like to test whenever this is available :) I'm excited to see if vscode can become more of a first-class editor to Godot with these debugger changes! I just recently rebased some changes that hopefully should make the language server more feature packed as well (https://github.com/godotengine/godot/pull/80973)

Let me know when testing begins!

[Calinou]

If you want to test this PR's current state, install the VSIX contained in this ZIP archive in VS Code (see instructions):

• https://nightly.link/godotengine/godot-vscode-plugin/actions/artifacts/846981313.zip

[snkaupe]

I'm unable to launch a debugging session using the provided VSIX. Every attempt to do so results in the following error message:

Failed to launch Godot instance: Error: Command failed: "godot" --path "<project-path>" --remote-debug "tcp://tcp://127.0.0.1:6007"
Der Befehl ""godot"" ist entweder falsch geschrieben oder
konnte nicht gefunden werden.

The part in German says that the executable "godot" was not found. Which is not surprising, because it is not part of my PATH variable. I did set the value for the new Godot 4 Editor setting, however, in the GUI settings editor, it is always reverted to undefined. It is, however, found and set correctly in the settings.json:

"godotTools.lsp.serverPort": 6005,
"godotTools.editorPath.godot4": "E:\\Entwicklung\\Godot\\Godot_v4.1.1-stable_win64.exe"

Additionally, when I try to exit debugging mode by clicking the "Stop" button, the following error message appears and I have to click it a second time to make the debug bar disappear:

ReferenceError: TerminatedEvent is not defined
    at Oc.terminateRequest (c:\Users\Sebastian\.vscode\extensions\geequlim.godot-tools-1.3.1\out\extension.js:274:81967)
    at Oc.dispatchRequest (c:\Users\Sebastian\.vscode\extensions\geequlim.godot-tools-1.3.1\out\extension.js:51:5822)
    at Oc.dispatchRequest (c:\Users\Sebastian\.vscode\extensions\geequlim.godot-tools-1.3.1\out\extension.js:57:2045)
    at Oc.handleMessage (c:\Users\Sebastian\.vscode\extensions\geequlim.godot-tools-1.3.1\out\extension.js:45:7461)
    at C.sendMessage (e:\Entwicklung\Microsoft VS Code\resources\app\out\vs\workbench\api\node\extensionHostProcess.js:119:705)
    at c.$sendDAMessage (e:\Entwicklung\Microsoft VS Code\resources\app\out\vs\workbench\api\node\extensionHostProcess.js:118:9830)
    at t.N (e:\Entwicklung\Microsoft VS Code\resources\app\out\vs\workbench\api\node\extensionHostProcess.js:114:7983)
    at t.M (e:\Entwicklung\Microsoft VS Code\resources\app\out\vs\workbench\api\node\extensionHostProcess.js:114:7749)
    at t.H (e:\Entwicklung\Microsoft VS Code\resources\app\out\vs\workbench\api\node\extensionHostProcess.js:114:6830)
    at t.G (e:\Entwicklung\Microsoft VS Code\resources\app\out\vs\workbench\api\node\extensionHostProcess.js:114:5906)
    at r.value (e:\Entwicklung\Microsoft VS Code\resources\app\out\vs\workbench\api\node\extensionHostProcess.js:114:4736)
    at f.w (e:\Entwicklung\Microsoft VS Code\resources\app\out\vs\workbench\api\node\extensionHostProcess.js:63:1902)
    at f.fire (e:\Entwicklung\Microsoft VS Code\resources\app\out\vs\workbench\api\node\extensionHostProcess.js:63:2119)
    at y.fire (e:\Entwicklung\Microsoft VS Code\resources\app\out\vs\workbench\api\node\extensionHostProcess.js:79:13931)
    at r.value (e:\Entwicklung\Microsoft VS Code\resources\app\out\vs\workbench\api\node\extensionHostProcess.js:130:30355)
    at f.w (e:\Entwicklung\Microsoft VS Code\resources\app\out\vs\workbench\api\node\extensionHostProcess.js:63:1902)
    at f.fire (e:\Entwicklung\Microsoft VS Code\resources\app\out\vs\workbench\api\node\extensionHostProcess.js:63:2119)
    at y.fire (e:\Entwicklung\Microsoft VS Code\resources\app\out\vs\workbench\api\node\extensionHostProcess.js:79:13931)
    at MessagePortMain.<anonymous> (e:\Entwicklung\Microsoft VS Code\resources\app\out\vs\workbench\api\node\extensionHostProcess.js:130:28635)
    at MessagePortMain.emit (node:events:513:28)
    at MessagePortMain._internalPort.emit (node:electron/js2c/utility_init:2:367)

[ryanabx]

I get this error when trying to debug and run from vscode:

and when trying to end the failed debug session:

[DaelonSuzuka]

New things that work:

• Godot4 launch mode
  • breakpoints + continuing
  • stack dump
  • variable inspection

New things that are broken:

• Godot 4 scene tree
• all editorPath settings
• maybe more idk

Video evidence:

[Haffi921]

Video evidence: ...

@DaelonSuzuka I downloaded and packaged this PR but I still get the same non-functionality that I get with the normal extension. Could you share the launch.json configuration for godot4 you have in this video demo?

[DaelonSuzuka]

@Haffi921 this should be it:

{
    "version": "0.2.0",
    "configurations": [
        {
            "name": "Launch",
            "type": "godot4",
            "request": "launch",
            "editor_path": "godot4",
            "project": "${workspaceFolder}"
        },
        {
            "name": "Attach",
            "type": "godot4",
            "request": "attach"
        }
    ]
}

[DaelonSuzuka]

Lots of improvements:

• rebased on master, bringing in some important changes
• identified an important change to the message format in g4
• terminating launch sessions should ACTUALLY work now

[DaelonSuzuka]

Progress report:

Debug profiles have been unified back into a single godot profile. Your launch.json will need to be updated.

Attach sessions don't work correctly in either version, don't bother testing these.

Launch sessions in 3 and 4 should both mostly work:

• starting and stopping the session finally works correctly
• breakpoints in 4 seem to mostly work (continue, step, etc)
• breakpoints in 3 seem to work ONCE, after continuing the first breakpoint it doesn't stop again
• stack dump on breakpoint works on both
• variable inspection per stack frame works on 4 but not on 3
• scene tree works on 3 but not 4
• node inspector doesn't work on either

[DaelonSuzuka]

All the major features work, in both Godot versions:

• breakpoints
• stepping, continuing, stopping
• stack frames
• variables inspection
• scene tree
• inspector

Debug configuration has been totally overhauled. This is the launch.json I recorded the following gifs with:

{
    "version": "0.2.0",
    "configurations": [
        {
            "name": "Launch",
            "type": "godot",
        }
    ]
}

387 has been implemented in this PR (Godot 4 only). Shoutout to @ZachIsAGardner

386 has been implemented in the PR. Double shoutout to Zach

[Geometror]

Great work so far! Tested this with the godot-benchmarks project (both with and without addtional arguments).

{
    "name": "GDScript: Launch Godot",
    "type": "godot",
    "request": "launch",
    "project": "${workspaceFolder}",
    "additional_options": "-- --run-benchmarks --include-benchmarks=\"rendering/culling/\""
}

Unfortunately, I can't get the project to launch: There is no output in "Extension Host", just when loading the extension:

[DaelonSuzuka]

Thanks for the report, @Geometror, Can you try again with the commit I just pushed?

[Geometror]

The project starts now, and debugging was possible once (breakpoints, pausing) but for some reason I couldn't get it to work a second time (even after restarting the extension host, reinstalling, restarting VSCode) I still get this regularly, even after the extension has been loaded:

And sometimes this:

Also during the one time it worked I noticed that the entries in the active scene tree were missing their icons. (they were present in a GIF you posted earlier)

[DaelonSuzuka]

The project starts now, and debugging was possible once (breakpoints, pausing) but for some reason I couldn't get it to work a second time (even after restarting the extension host, reinstalling, restarting VSCode)

Progress! Just for the sake of completeness, what OS and Godot version are you using?

I still get this regularly, even after the extension has been loaded:

Hopefully this is fixed now.

And sometimes this:

I think this is why it only worked once. I think the socket isn't being closed properly and it's still active the next time you try to launch. I'm already investigating some resource cleanup issues, hopefully I can track this down pretty quick.

Also during the one time it worked I noticed that the entries in the active scene tree were missing their icons. (they were present in a GIF you posted earlier)

Interesting. I'll try to add some logging around the scene tree so maybe we can find out why that's happening.

[Geometror]

Both on my Windows 10 and Linux machine, with both Godot 4.1.2 and 4.2 beta 4 the project starts, but after that there is no connection to the debugger. (tested with your latest changes)

[Fox-Alpha]

Both on my Windows 10 and Linux machine, with both Godot 4.1.2 and 4.2 beta 4 the project starts, but after that there is no connection to the debugger. (tested with your latest changes)

I have tested the latest artifact from build with Windows 10 and 4.2 Beta 4-mono The Engine wrote this Message in Console

editor/debugger/debug_adapter/debug_adapter_protocol.cpp:800 - Condition "!breakpoint_list.find(breakpoint)" is true. Returning: Array()

VSCode do not stop at breakpoints

Greetings Fox

[DaelonSuzuka]

@Geometror

The EADDRINUSE error definitely shouldn't be happening anymore.

@Fox-Alpha

The Engine wrote this Message in Console

editor/debugger/debug_adapter/debug_adapter_protocol.cpp:800 - Condition "!breakpoint_list.find(breakpoint)" is true. Returning: Array()

Connecting to the Godot editor is currently not supported, so that's not surprising.

[Geometror]

Unfortunately I still can't walk through breakpoints/pause the project. As soon as I set any breakpoint the project freezes on the splash screen. (W11, GD4.1)

Besides that, when your godot executable path is not set correctly the extension just fails silently (the debugger control panel opens, but the buttons don't do anything and there is no warning). In addition to that, I would set the default Godot 4.x editor path to godot instead of godot4, since most users need to adjust this setting.

[DaelonSuzuka]

Besides that, when your godot executable path is not set correctly the extension just fails silently (the debugger control panel opens, but the buttons don't do anything and there is no warning). In addition to that, I would set the default Godot 4.x editor path to godot instead of godot4, since most users need to adjust this setting.

I've actually been thinking about setting the default to "" and forcing the user to enter it on first run using a file picker or something.

screenshot

I really wish I could see the message it received that's causing that error...

I'm going to add some error handling and try to redirect my internal logging to an output channel, so it's accessible at runtime.

In the meantime, would it be possible for you to zip up your project and send it to me somehow?

[Geometror]

I've actually been thinking about setting the default to "" and forcing the user to enter it on first run using a file picker or something.

Having a dialog is a good idea I think. We could check whether godot is in PATH and if not show the dialog. (you could even determine the version via parsing the output of godot --version, but that could be implemented later)

I'm going to add some error handling and try to redirect my internal logging to an output channel, so it's accessible at runtime.

That would be great, I think this extension should have its own output channel anyway.

In the meantime, would it be possible for you to zip up your project and send it to me somehow?

I just cloned https://github.com/godotengine/godot-benchmarks and added the default launch configuration.

[DaelonSuzuka]

Having a dialog is a good idea I think. We could check whether godot is in PATH and if not show the dialog. (you could even determine the version via parsing the output of godot --version, but that could be implemented later)

That entire process is already being done for the headless LSP startup, so it'll take me 5 minutes to copy it over. That's actually why I hadn't done it yet: I already know that it works.

I just cloned godotengine/godot-benchmarks and added the default launch configuration.

That's actually awesome, thank you.

[samjw-nz]

Does this PR accommodate issue 462? I (along with others) found the linked fix worked, so I imagine changing the default address to "tcp//127.0.0.1" would prevent some issues for future users. https://github.com/godotengine/godot-vscode-plugin/issues/462#issuecomment-1497566405

[atirut-w]

I think a better solution would be automatically prepending tcp://.

[DaelonSuzuka]

Does this PR accommodate issue 462?

Yes, even though I've tested this in a variety of situations and adding tcp:// has never made any difference as far as I can tell. It also doesn't hurt anything, so I added it anyways.

Although, more importantly, the stupid address field is now optional, like it always should have been.

[DaelonSuzuka]

@Geometror

Unfortunately I still can't walk through breakpoints/pause the project. As soon as I set any breakpoint the project freezes on the splash screen.

Apparently the debugger message format changed between 4.1 and 4.2. It was an easy fix now that I found it.

Breakpoints also weren't working for me in the benchmark repo because I added -- --run-benchmarks to additional_options in the launch configuration. The additional_options was being added BEFORE the breakpoint string, and the -- forces everything that comes after to be ignored by the engine and passed to the game. I fixed it so that additional_options is now always last, and breakpoints are working in both versions (for me).

[DaelonSuzuka]

Completed a bunch of user-facing polish. Error handling, prompts/dialogs for remediation, logging goes to an output panel (only for the debugger so far).

I've removed the commands Run workspace as Godot project and Run workspace as Godot project with visible collision shapes and navigation meshes because they're just a worse version of launching a debug session.

Open workspace with Godot editor is staying, and it's been rewritten to A) not be completely illegible, and B) validate the workspace version against the Godot version it's about to launch + handle other errors.

Log output screenshot:

[Geometror]

Btw is this PR's title still aptly? This looks more like a complete rewrite than just "debugger fixes" :)

[DaelonSuzuka]

Works like a charm now

Thanks for taking the time to test this branch, your feedback has been extremely valuable!

Btw is this PR's title still aptly? This looks more like a complete rewrite than just "debugger fixes" :)

We usually fix the PR titles right before merging.

[rsubtil]

The only thing still missing are the icons in the active screen tree view, but that's not really a blocker:

Regarding this, would it make sense to implement some functionality in Godot for debuggers to request editor icons, like a request using the existing debugging protocol to fetch a particular icon? I think it's a more reliable solution than to bundle the editor's icons like it is currently. If you think this is useful lemme know and I can start a proposal to see what the community's opinion is.

[DaelonSuzuka]

That's a good idea, but it makes me concerned about the volume of data we'd end up shuffling around. There's already an issue with slowdowns after a breakpoint hits and we have to recursively fetch inspections for all the variables.

Also thanks for reminding me that I need to update the icons!

[atirut-w]

Possibly offtopic, but this hyped me up so much. Can't wait for the next release!