GDB session ended. but the debug toolbar won't close.

[yueduz]

Screenshots

Environment (please complete the following information):

[comment]: <> Whenever possible, please make sure you are using the latest versions of VSCode and our extension

• Cortex-Debug Version (this extension):v1.12.1
• OS:archlunux
• GDB Version: [e.g. 1.11.1]
• Compiler Toolchain Version: arm-none-eabi 14.1.0

Please include launch.json

{
    // 使用 IntelliSense 了解相关属性。 
    // 悬停以查看现有属性的描述。
    // 欲了解更多信息，请访问: https://go.microsoft.com/fwlink/?linkid=830387
    "version": "0.2.0",
    "configurations": [
        {
            "name": "Debug Jlink",
            "type": "cortex-debug",
            "request": "launch",
            "cwd": "${workspaceRoot}",
            "executable": "${workspaceRoot}/build/debug/AT32_Audio.elf",
            "servertype": "jlink",
            "device": "AT32F435CMU7",
            "interface": "swd",
            // "runToEntryPoint": "main",
            // "overrideLaunchCommands": [
            //     "monitor halt",
            //     "monitor reset",
            //     "-target-download",
            //     "-enable-pretty-printing"
            // ],
            // "overrideRestartCommands": [
            //     "monitor reset",
            //     "-enable-pretty-printing"
            // ],
            // "showDevDebugOutput": "raw" ,
            "svdFile": "${workspaceRoot}/AT32F435CMU7_WorkBench/project/AT32_IDE/SVD/AT32F435xx_v2.svd",
            "preLaunchTask": "build Debug", //先运行make任务
            "serverArgs": [
                "-nologtofile",
                "-s",
                "-vd",
                "-speed",
                "15000"
            ],
            "rttConfig": {
                "enabled": false,
                "address": "auto",
                // "clearSearch": false    // OpenOCD users may have to un-comment this
                "decoders": [
                    {
                        "port": 0,
                        "type": "console"
                    }
                ]
            }
        }
    ]
}

Attach text from Debug Console

Please enable debug output in your launch.json ("showDevDebugOutput": "raw"). It this is too large, please attach it as a file

JLinkGDBServer -singlerun -nogui -if swd -port 50000 -swoport 50001 -telnetport 50002 -device AT32F435CMU7 -nologtofile -s -vd -speed 15000
SEGGER J-Link GDB Server V7.96q Command Line Version

JLinkARM.dll V7.96q (DLL compiled Jun 28 2024 12:39:24)

Command line: -singlerun -nogui -if swd -port 50000 -swoport 50001 -telnetport 50002 -device AT32F435CMU7 -nologtofile -s -vd -speed 15000
-----GDB Server start settings-----
GDBInit file:                  none
GDB Server Listening port:     50000
SWO raw output listening port: 50001
Terminal I/O port:             50002
Accept remote connection:      yes
Generate logfile:              off
Verify download:               on
Init regs on start:            off
Silent mode:                   off
Single run mode:               on
Target connection timeout:     0 ms
------J-Link related settings------
J-Link Host interface:         USB
J-Link script:                 none
J-Link settings file:          none
------Target related settings------
Target device:                 AT32F435CMU7
Target device parameters:      none
Target interface:              SWD
Target interface speed:        15000kHz
Target endian:                 little

Connecting to J-Link...
J-Link is connected.
Firmware: J-Link V10 compiled Jan 30 2023 11:28:07
Hardware: V10.10
S/N: 50116405
Feature(s): GDB
Checking target voltage...
Target voltage: 3.32 V
Listening on TCP/IP port 50000
Connecting to target...
Halting core...
Connected to target
Waiting for GDB connection...Connected to 0000:0000:0000:0000:0000:0000:0000:0001
GDB client (conn. 12) requested target.xml from GDB Server
Reading common registers: Read register 'r0' (4 bytes) from hardware: 0x5C8D0520
Read register 'r1' (4 bytes) from hardware: 0x64FF0520
Read register 'r2' (4 bytes) from hardware: 0x05000000
Read register 'r3' (4 bytes) from hardware: 0x00000000
Read register 'r4' (4 bytes) from hardware: 0x5C8D0520
Read register 'r5' (4 bytes) from hardware: 0x78860520
Read register 'r6' (4 bytes) from hardware: 0x64FF0520
Read register 'r7' (4 bytes) from hardware: 0x00000000
Read register 'r8' (4 bytes) from hardware: 0x00000000
Read register 'r9' (4 bytes) from hardware: 0x00000000
Read register 'r10' (4 bytes) from hardware: 0x00000000
Read register 'r11' (4 bytes) from hardware: 0x00000000
Read register 'r12' (4 bytes) from hardware: 0x00000000
Read register 'sp' (4 bytes) from hardware: 0xF8FE0520
Read register 'lr' (4 bytes) from hardware: 0xE1FFFFFF
Read register 'pc' (4 bytes) from hardware: 0xBE990008
Read register 'xpsr' (4 bytes) from hardware: 0x03000061
Read 4 bytes @ address 0x080099BE (Data = 0xE7FEE7FE)
Received monitor command: halt
Halting target CPU...
...Target halted (PC = 0x080099BE)
Received monitor command: reset
Resetting target
Downloading 524 bytes @ address 0x08000000 - Verified OK
Downloading 16000 bytes @ address 0x08000210 - Verified OK
Downloading 16000 bytes @ address 0x08004090 - Verified OK
Downloading 16096 bytes @ address 0x08007F10 - Verified OK
Downloading 3240 bytes @ address 0x0800BDF0 - Verified OK
Downloading 6676 bytes @ address 0x0800CA98 - Verified OK
Downloading 8 bytes @ address 0x0800E4AC - Verified OK
Downloading 4 bytes @ address 0x0800E4B4 - Verified OK
Downloading 4 bytes @ address 0x0800E4B8 - Verified OK
Downloading 868 bytes @ address 0x0800E4BC - Verified OK
Writing register 'pc' = 0x08000340
Comparing flash   [....................] Done.
Erasing flash     [....................] Done.
Programming flash [....................] Done.
Verifying flash   [....................] Done.
Received monitor command: reset
Resetting target
Reading 64 bytes @ address 0x08009A00
Read 2 bytes @ address 0x08009A2C (Data = 0xF7FE)
Reading 64 bytes @ address 0x08008B00
Read 2 bytes @ address 0x08008B94 (Data = 0xED96)
Reading 64 bytes @ address 0x08009200
Read 2 bytes @ address 0x08009206 (Data = 0xF44F)
Read 4 bytes @ address 0x50000E00 (Data = 0x00000000)
Setting breakpoint @ address 0x08008B94, Kind = 2, Type = THUMB, BPHandle = 0x0001
Setting breakpoint @ address 0x08009206, Kind = 2, Type = THUMB, BPHandle = 0x0002
Setting breakpoint @ address 0x08009A2C, Kind = 2, Type = THUMB, BPHandle = 0x0003
Starting target CPU...
ERROR: Cannot read register 15 (R15) while CPU is running
Reading common registers: ERROR: Cannot read register 0 (R0) while CPU is running
Read register 'r0' (4 bytes) from hardware: 0xEFBEADDE
ERROR: Cannot read register 1 (R1) while CPU is running
Read register 'r1' (4 bytes) from hardware: 0xEFBEADDE
ERROR: Cannot read register 2 (R2) while CPU is running
Read register 'r2' (4 bytes) from hardware: 0xEFBEADDE
ERROR: Cannot read register 3 (R3) while CPU is running
Read register 'r3' (4 bytes) from hardware: 0xEFBEADDE
ERROR: Cannot read register 4 (R4) while CPU is running
Read register 'r4' (4 bytes) from hardware: 0xEFBEADDE
ERROR: Cannot read register 5 (R5) while CPU is running
Read register 'r5' (4 bytes) from hardware: 0xEFBEADDE
ERROR: Cannot read register 6 (R6) while CPU is running
Read register 'r6' (4 bytes) from hardware: 0xEFBEADDE
ERROR: Cannot read register 7 (R7) while CPU is running
Read register 'r7' (4 bytes) from hardware: 0xEFBEADDE
ERROR: Cannot read register 8 (R8) while CPU is running
Read register 'r8' (4 bytes) from hardware: 0xEFBEADDE
ERROR: Cannot read register 9 (R9) while CPU is running
Read register 'r9' (4 bytes) from hardware: 0xEFBEADDE
ERROR: Cannot read register 10 (R10) while CPU is running
Read register 'r10' (4 bytes) from hardware: 0xEFBEADDE
ERROR: Cannot read register 11 (R11) while CPU is running
Read register 'r11' (4 bytes) from hardware: 0xEFBEADDE
ERROR: Cannot read register 12 (R12) while CPU is running
Read register 'r12' (4 bytes) from hardware: 0xEFBEADDE
ERROR: Cannot read register 13 (R13) while CPU is running
Read register 'sp' (4 bytes) from hardware: 0xEFBEADDE
ERROR: Cannot read register 14 (R14) while CPU is running
Read register 'lr' (4 bytes) from hardware: 0xEFBEADDE
ERROR: Cannot read register 15 (R15) while CPU is running
Read register 'pc' (4 bytes) from hardware: 0xEFBEADDE
ERROR: Cannot read register 16 (XPSR) while CPU is running
Read register 'xpsr' (4 bytes) from hardware: 0xEFBEADDE
Removing breakpoint @ address 0x08008B94, Size = 2
Removing breakpoint @ address 0x08009206, Size = 2
Removing breakpoint @ address 0x08009A2C, Size = 2
WARNING: Failed to read memory @ address 0xDEADBEEE
ERROR: Cannot read register 17 (MSP) while CPU is running
Reading register 'msp' = 0xDEADBEEF
ERROR: Cannot read register 18 (PSP) while CPU is running
Reading register 'psp' = 0xDEADBEEF
WARNING: Failed to read memory @ address 0xDEADBEEE
WARNING: Failed to read memory @ address 0x50000E00
WARNING: Failed to read memory @ address 0x50000E00
WARNING: Failed to read memory @ address 0x50000E03
WARNING: Target connection lost.
Received monitor command: halt
Halting target CPU...
...Target halted (PC = 0x00000000)
Received monitor command: reset
Resetting target
Setting breakpoint @ address 0x08008B94, Kind = 2, Type = THUMB, BPHandle = 0x0000
WARNING: No more breakpoint resources left
ERROR: Failed to set breakpoint at 0x08008B94
Setting breakpoint @ address 0x08009206, Kind = 2, Type = THUMB, BPHandle = 0x0000
WARNING: No more breakpoint resources left
ERROR: Failed to set breakpoint at 0x08009206
Setting breakpoint @ address 0x08009A2C, Kind = 2, Type = THUMB, BPHandle = 0x0000
WARNING: No more breakpoint resources left
ERROR: Failed to set breakpoint at 0x08009A2C
WARNING: Failed to read memory @ address 0xDEADBEEE
WARNING: Failed to read memory @ address 0xDEADBEEE
WARNING: Failed to read memory @ address 0x50000E00
WARNING: Failed to read memory @ address 0x50000E00
WARNING: Failed to read memory @ address 0x50000E03
Received monitor command: halt
Halting target CPU...
...Target halted (PC = 0x080078D6)
Received monitor command: reset
Resetting target
Setting breakpoint @ address 0x08008B94, Kind = 2, Type = THUMB, BPHandle = 0x0004
Setting breakpoint @ address 0x08009206, Kind = 2, Type = THUMB, BPHandle = 0x0005
Setting breakpoint @ address 0x08009A2C, Kind = 2, Type = THUMB, BPHandle = 0x0006
Starting target CPU...
Debugger requested to halt target...
...Target halted (PC = 0x080078D6)
Reading common registers: Read register 'r0' (4 bytes) from hardware: 0x00000000
Read register 'r1' (4 bytes) from hardware: 0x00000000
Read register 'r2' (4 bytes) from hardware: 0x05000000
Read register 'r3' (4 bytes) from hardware: 0x01000000
Read register 'r4' (4 bytes) from hardware: 0x708D0520
Read register 'r5' (4 bytes) from hardware: 0x00000000
Read register 'r6' (4 bytes) from hardware: 0x00000000
Read register 'r7' (4 bytes) from hardware: 0x00000000
Read register 'r8' (4 bytes) from hardware: 0x00000000
Read register 'r9' (4 bytes) from hardware: 0x00000000
Read register 'r10' (4 bytes) from hardware: 0x00000000
Read register 'r11' (4 bytes) from hardware: 0x00000000
Read register 'r12' (4 bytes) from hardware: 0x02000000
Read register 'sp' (4 bytes) from hardware: 0xF8FF0520
Read register 'lr' (4 bytes) from hardware: 0xE7A40008
Read register 'pc' (4 bytes) from hardware: 0xD6780008
Read register 'xpsr' (4 bytes) from hardware: 0x00000061
Removing breakpoint @ address 0x08008B94, Size = 2
Removing breakpoint @ address 0x08009206, Size = 2
Removing breakpoint @ address 0x08009A2C, Size = 2
Read 4 bytes @ address 0x080078D6 (Data = 0x23E0F04F)
Reading register 'msp' = 0x2005FFF8
Reading register 'psp' = 0x00000000
Read 4 bytes @ address 0x0800A4E6 (Data = 0x6430E7FB)
Reading 64 bytes @ address 0x2005FFC0
Read 4 bytes @ address 0x50000E00 (Data = 0x00000001)
Received monitor command: halt
Halting target CPU...
...Target halted (PC = 0x080078D6)
Received monitor command: reset
Resetting target
Setting breakpoint @ address 0x08008B94, Kind = 2, Type = THUMB, BPHandle = 0x0007
Setting breakpoint @ address 0x08009206, Kind = 2, Type = THUMB, BPHandle = 0x0008
Setting breakpoint @ address 0x08009A2C, Kind = 2, Type = THUMB, BPHandle = 0x0009
Starting target CPU...
Debugger requested to halt target...
...Target halted (PC = 0x080078DC)
Reading common registers: Read register 'r0' (4 bytes) from hardware: 0x00000000
Read register 'r1' (4 bytes) from hardware: 0x00000000
Read register 'r2' (4 bytes) from hardware: 0x05000000
Read register 'r3' (4 bytes) from hardware: 0x05000000
Read register 'r4' (4 bytes) from hardware: 0x708D0520
Read register 'r5' (4 bytes) from hardware: 0x00000000
Read register 'r6' (4 bytes) from hardware: 0x00000000
Read register 'r7' (4 bytes) from hardware: 0x00000000
Read register 'r8' (4 bytes) from hardware: 0x00000000
Read register 'r9' (4 bytes) from hardware: 0x00000000
Read register 'r10' (4 bytes) from hardware: 0x00000000
Read register 'r11' (4 bytes) from hardware: 0x00000000
Read register 'r12' (4 bytes) from hardware: 0x02000000
Read register 'sp' (4 bytes) from hardware: 0xF8FF0520
Read register 'lr' (4 bytes) from hardware: 0xE7A40008
Read register 'pc' (4 bytes) from hardware: 0xDC780008
Read register 'xpsr' (4 bytes) from hardware: 0x00000061
Removing breakpoint @ address 0x08008B94, Size = 2
Removing breakpoint @ address 0x08009206, Size = 2
Removing breakpoint @ address 0x08009A2C, Size = 2
Read 4 bytes @ address 0x080078DC (Data = 0x1301F003)
Reading register 'msp' = 0x2005FFF8
Reading register 'psp' = 0x00000000
Read 4 bytes @ address 0x0800A4E6 (Data = 0x6430E7FB)
Reading 64 bytes @ address 0x2005FFC0
Received monitor command: halt
Halting target CPU...
...Target halted (PC = 0x080078DC)
Received monitor command: reset
Resetting target
Setting breakpoint @ address 0x08008B94, Kind = 2, Type = THUMB, BPHandle = 0x000A
Setting breakpoint @ address 0x08009206, Kind = 2, Type = THUMB, BPHandle = 0x000B
Setting breakpoint @ address 0x08009A2C, Kind = 2, Type = THUMB, BPHandle = 0x000C
Starting target CPU...
Debugger requested to halt target...
...Target halted (PC = 0x080078DA)
Reading common registers: Read register 'r0' (4 bytes) from hardware: 0x00000000
Read register 'r1' (4 bytes) from hardware: 0x00000000
Read register 'r2' (4 bytes) from hardware: 0x05000000
Read register 'r3' (4 bytes) from hardware: 0x00E000E0
Read register 'r4' (4 bytes) from hardware: 0x708D0520
Read register 'r5' (4 bytes) from hardware: 0x00000000
Read register 'r6' (4 bytes) from hardware: 0x00000000
Read register 'r7' (4 bytes) from hardware: 0x00000000
Read register 'r8' (4 bytes) from hardware: 0x00000000
Read register 'r9' (4 bytes) from hardware: 0x00000000
Read register 'r10' (4 bytes) from hardware: 0x00000000
Read register 'r11' (4 bytes) from hardware: 0x00000000
Read register 'r12' (4 bytes) from hardware: 0x02000000
Read register 'sp' (4 bytes) from hardware: 0xF8FF0520
Read register 'lr' (4 bytes) from hardware: 0xE7A40008
Read register 'pc' (4 bytes) from hardware: 0xDA780008
Read register 'xpsr' (4 bytes) from hardware: 0x00000061
Removing breakpoint @ address 0x08008B94, Size = 2
Removing breakpoint @ address 0x08009206, Size = 2
Removing breakpoint @ address 0x08009A2C, Size = 2
Read 4 bytes @ address 0x080078DA (Data = 0xF003691B)
Reading register 'msp' = 0x2005FFF8
Reading register 'psp' = 0x00000000
Read 4 bytes @ address 0x0800A4E6 (Data = 0x6430E7FB)
Reading 64 bytes @ address 0x2005FFC0
GDB closed TCP/IP connection (Socket 12)
Restoring target state and closing J-Link connection...
Shutting down...
[2024-07-03T01:42:59.498Z] SERVER CONSOLE DEBUG: onBackendConnect: gdb-server session closed
GDB server session ended. This terminal will be reused, waiting for next session to start...

Program
 received signal SIGTRAP, Trace/breakpoint trap.
delay_ms (nms=0x0, nms@entry=0x2) at /home/lei/at32-work-bench/AT32_Audio/AT32F435CMU7_WorkBench/mylib/delay/delay.c:59
59      } while ((temp & 0x01) && !(temp & (1 << 16)));
Resetting target

Program
 received signal SIGTRAP, Trace/breakpoint trap.
0x080078da in delay_ms (nms=0x0, nms@entry=0x2) at /home/lei/at32-work-bench/AT32_Audio/AT32F435CMU7_WorkBench/mylib/delay/delay.c:58
58        temp = SysTick->CTRL;
GDB session ended. exit-code: 0

[haneefdm]

Please enable debug output in your launch.json ("showDevDebugOutput": "raw"). It this is too large, please attach it as a file

See, you did not do that, so I have nothing to look at. Nothing changed on our end in almost six months. Did JLink change? Did you try rolling back a few versions. I really doubt JLink caused an issue but, did you look at what could have gone wrong?

The toolbar does not change state because VSCode (it controls that toolbar) thinks the debugger is still running. You also did not say what you did to get the debugger into this state. How am I supposed to reproduce/debug this?

I'm closing this until you give me a better description.

[RolfNoot]

Adding some info here, experiencing the same issue since the last update (probably with 1.12, but not 100% sure).

Replication of the issue is easy:

1. start debug
2. wait at main breakpoint
3. press restart
4. press stop/disconnect: GDB session ends but toolbar remains

See video: https://github.com/Marus/cortex-debug/assets/44534542/dc8668cf-562b-49d8-b0e3-f70b4f5eb5e5

See debug info: terminal_output.txt (not much to see here as the issue happens AFTER the debug session ends).

[haneefdm]

@RolfNoot , that indicates a crash in JS code. If you click on that popup, you may see more info on the source or the problem.

In your system temp directory (indicated by $TMPDIR on Mac/Linux and %TEMP% env. var on Windows), hopefully in that directory there is a filed called cortex-debug-server-exiting.log. Please use your native shell to find this file and not something like cygwin/msys. It may contain some useful info as well. I only need the last 100 lines from this file. This file is created by a non GUI backed program and may indicate it crashing instead of the frontend GUI. The backend seems to have exited cleanly, but you never know. Because VSCode is still waiting for the backend to quit which is why the toolbar is in the wrong state.

[haneefdm]

Please see https://github.com/Marus/cortex-debug/wiki/Cortex-Debug-Under-the-hood on how a typical debugger is put together.

[RolfNoot]

This is the log file cortex-debug-server-exiting.log

This is the output of the Developer Tools console as soon as the server disconnects but the toolbar remains:

I am also debugging your extension:

Let me know if you need me do check something while debugging, put breakpoints or need anything more

[haneefdm]

I am on holiday (Independence Day in US). Besides, I don't have a board yet. Expecting one soon. Get back to this later.

It is one thing for gdb to exit, but another for us to exit because there is other cleanup that has to occur. It appears that messages are being generated even though the debugger has crashed or destroyed. Need to go up the stack to see what is going on.

@RolfNoot Have you read my Wiki page I attached above? Of course, you are the expert!

One of the problems is that the Heisenberg principle comes in by putting breakpoints. Especially while ending the session. Your best bet is to put temporary debug messages in suspected areas like I tried to do with the cortex-debug-server-exiting.log file. I created the ServerConsoleLog(...) to help me.

[RolfNoot]

Hi @haneefdm,

Yes, I've read the wiki. I understand the issues you're facing and that it's even harder to pinpoint the eventual error, especially with 'vague' or limited user comments. I always feel that when the issue is to be reproduced, the solution is near.

Yes putting breakpoints can confuse even more and may completely break the functionality, especially when dealing with multithreaded designs. I mean if you want me to add anything to the log or help, I am here.

We're dealing with users as well and it's not always easy to help especially when there's not much information.

I am willing to send you a Onethinx LoRaWAN kit which contains PSoC6 if it may help speed up investigations.

Enjoy the free day!

[haneefdm]

I have a bad feeling about this. It appears that it is VSCode that is totally confused.

Over the last 5+ years, the definition and behavior of a 'Restart' kept changing. Maybe it changed again. At one point they did not trigger preLaunchTask on a Restart. Then they did a preLaunchTask. Then reverted back. Then reverted that decision. No idea what else changed with the Restart. The preLaunchTask is just one example.

A few years ago, we got tired of this and added a 'Reset' button because....

• In a lot of cases users did not want a re-build because they could take a LONG time even if nothing changed
• The functionality they desired was actually just a 'lets start from the beginning' without compiling or programming.

Thus, the Reset is fast and what most embedded developers wanted.

It appears from the logs that we have done everything we are supposed to do for a 'Restart' and Disconnect/Stop. But maybe their expectations changed. Need to focus on 'Restart' because it appears that if this was never called, it seems to work fine. Need to find out how debuggers are supposed to respond to a 'Restart' request. Rarely, documented so we have to do this by trial and error and looking at VSCode source. Perhaps play with cppdbg or some other debugger.

See also https://community.platformio.org/t/debugging-why-does-the-green-restart-icon-always-rebuild-entire-project/27410

[haneefdm]

Fixed it. VSCode was seriously messed up. I found many debuggers have removed the 'Restart' functionality from their adapters. Instead, VSCode totally restarts from scratch almost like you did a 'Stop' followed by a 'Start'. But I don't think it is re-reading the launch.json

Good riddance, because no one knew what the rules were.

But, now we have to remove Restart options from the launch.json (our package.json) because a Restart is now handled by VSCode and we have no involvement in it.

[RolfNoot]

Good to hear!

I was just close, looking at the differences between 1.4.4 which works and 1.5.0 which doesn't. If you like I can check further to pinpoint the causing issue.

[RolfNoot]

Fixed it. VSCode was seriously messed up. I found many debuggers have removed the 'Restart' functionality from their adapters. Instead, VSCode totally restarts from scratch almost like you did a 'Stop' followed by a 'Start'. But I don't think it is re-reading the launch.json

Confirmed, it works at my side. It is actually re-reading launch.json and applying changes if there are in launch.json.

Thanks so much, this issue was bothering me for some time!

[haneefdm]

Cc: @AndriiLishchynskyi

Adrii, this may affect you. If you are still working on this, please see https://github.com/Marus/cortex-debug/blob/master/CHANGELOG.md

If there are general issues, could you give your feedback here? If you want to chat privately, you have my GitHub handle.

[cc-caixf]

hello, does anyone konw how to fix it?

[tim-hilt]

I tested it with the current master branch, looks good to me as well @haneefdm. I'm not deeply informed about the dev-progress, but seems save to release from my POV.