"Remote replied unexpectedly to 'vMustReplyEmpty'" with OpenOCD

marcusb commented 2 days ago

I am debugging a Marvell mw320 target with OpenOCD. The debugger startup fails with:

Failed to launch GDB: Remote replied unexpectedly to 'vMustReplyEmpty': PacketSize=4000;qXfer:memory-map:read-;qXfer:features:read+;qXfer:threads:read+;QStartNoAckMode+;vContSupported+ (from target-select extended-remote localhost:50000)

To Reproduce

Start a debug session with the target connected.

After starting OpenOCD and connecting to the remote debugger, this error is produced on the debug console, and it does not connect to the remote.

Expected behavior

It should connect to remote and allow me to debug the program.

Environment (please complete the following information):

Cortex-Debug Version: 1.12.1
OS: Debian Linux trixie/sid
GDB Version: 15.1
Compiler Toolchain Version: arm-none-eabi-gcc version 13.2.1 20231009 (15:13.2.rel1-2)
OpenOCD 0.12.0 (Debian)

Please include launch.json

{
    // Use IntelliSense to learn about possible attributes.
    // Hover to view descriptions of existing attributes.
    // For more information, visit: https://go.microsoft.com/fwlink/?linkid=830387
    "version": "0.2.0",
    "configurations": [
        {
            "name": "Cortex Debug",
            "cwd": "${workspaceFolder}",
            "executable": "${workspaceFolder}/build/sesame.axf",
            "request": "launch",
            "type": "cortex-debug",
            "runToEntryPoint": "main",
            "servertype": "openocd",
            "gdbPath": "/usr/bin/gdb-multiarch",
            "searchDir": [
                "${workspaceFolder}/app/marvell/WMSDK/mw320/sdk/tools/OpenOCD/interface",
                "${workspaceFolder}/app/marvell/WMSDK/mw320/sdk/tools/OpenOCD"
            ],
            "configFiles": [
                "${workspaceFolder}/app/marvell/WMSDK/mw320/sdk/tools/OpenOCD/interface/ftdi.cfg",
                "${workspaceFolder}/app/marvell/WMSDK/mw320/sdk/tools/OpenOCD/openocd.cfg"
            ],
            "openOCDLaunchCommands": [
                "init",
                "load ${workspaceFolder}/build/sesame.axf 0x10013d"
            ],
            "objdumpPath": "/usr/bin/arm-none-eabi-objdump",
            "showDevDebugOutput": "raw",

        }
    ]
}

Attach text from Debug Console

Cortex-Debug: VSCode debugger extension version 1.12.1 git(652d042). Usage info: https://github.com/Marus/cortex-debug#usage
"configuration": {
    "name": "Cortex Debug",
    "cwd": "/home/marcus/src/FreeRTOS",
    "executable": "/home/marcus/src/FreeRTOS/build/sesame.axf",
    "request": "launch",
    "type": "cortex-debug",
    "runToEntryPoint": "main",
    "servertype": "openocd",
    "gdbPath": "/usr/bin/gdb-multiarch",
    "searchDir": [
        "/home/marcus/src/FreeRTOS/app/marvell/WMSDK/mw320/sdk/tools/OpenOCD/interface",
        "/home/marcus/src/FreeRTOS/app/marvell/WMSDK/mw320/sdk/tools/OpenOCD"
    ],
    "configFiles": [
        "/home/marcus/src/FreeRTOS/app/marvell/WMSDK/mw320/sdk/tools/OpenOCD/interface/ftdi.cfg",
        "/home/marcus/src/FreeRTOS/app/marvell/WMSDK/mw320/sdk/tools/OpenOCD/openocd.cfg"
    ],
    "openOCDLaunchCommands": [
        "init",
        "load /home/marcus/src/FreeRTOS/build/sesame.axf 0x10013d"
    ],
    "objdumpPath": "/usr/bin/arm-none-eabi-objdump",
    "showDevDebugOutput": "raw",
    "__configurationTarget": 6,
    "gdbServerConsolePort": 60000,
    "pvtAvoidPorts": [],
    "chainedConfigurations": {
        "enabled": false
    },
    "debuggerArgs": [],
    "swoConfig": {
        "enabled": false,
        "decoders": [],
        "cpuFrequency": 0,
        "swoFrequency": 0,
        "source": "probe"
    },
    "rttConfig": {
        "enabled": false,
        "decoders": []
    },
    "graphConfig": [],
    "preLaunchCommands": [],
    "postLaunchCommands": [],
    "preAttachCommands": [],
    "postAttachCommands": [],
    "preRestartCommands": [],
    "postRestartCommands": [],
    "preResetCommands": [],
    "postResetCommands": [],
    "toolchainPrefix": "arm-none-eabi",
    "extensionPath": "/home/marcus/.vscode/extensions/marus25.cortex-debug-1.12.1",
    "registerUseNaturalFormat": true,
    "variableUseNaturalFormat": true,
    "pvtVersion": "1.12.1",
    "__sessionId": "a81e0bcf-2796-4dd9-b152-fc4c1bac2f10",
    "pvtShowDevDebugOutput": "raw"
}
Reading symbols from /usr/bin/arm-none-eabi-objdump --syms -C -h -w /home/marcus/src/FreeRTOS/build/sesame.axf
Reading symbols from /usr/bin/arm-none-eabi-nm --defined-only -S -l -C -p /home/marcus/src/FreeRTOS/build/sesame.axf
Launching GDB: /usr/bin/gdb-multiarch -q --interpreter=mi2
1-gdb-version
Launching gdb-server: openocd -c "gdb_port 50000" -c "tcl_port 50001" -c "telnet_port 50002" -s /home/marcus/src/FreeRTOS/app/marvell/WMSDK/mw320/sdk/tools/OpenOCD/interface -s /home/marcus/src/FreeRTOS/app/marvell/WMSDK/mw320/sdk/tools/OpenOCD -f /home/marcus/.vscode/extensions/marus25.cortex-debug-1.12.1/support/openocd-helpers.tcl -f /home/marcus/src/FreeRTOS/app/marvell/WMSDK/mw320/sdk/tools/OpenOCD/interface/ftdi.cfg -f /home/marcus/src/FreeRTOS/app/marvell/WMSDK/mw320/sdk/tools/OpenOCD/openocd.cfg -c init -c "load /home/marcus/src/FreeRTOS/build/sesame.axf 0x10013d"
    Please check TERMINAL tab (gdb-server) for output from openocd
Finished reading symbols from objdump: Time: 90 ms
-> =thread-group-added,id="i1"
-> ~"GNU gdb (Debian 15.1-1) 15.1\n"
-> ~"Copyright (C) 2024 Free Software Foundation, Inc.\n"
-> ~"License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>\nThis is free software: you are free to change and redistribute it.\nThere is NO WARRANTY, to the extent permitted by law."
-> ~"\nType \"show copying\" and \"show warranty\" for details.\n"
-> ~"This GDB was configured as \"x86_64-linux-gnu\".\n"
-> ~"Type \"show configuration\" for configuration details.\n"
-> ~"For bug reporting instructions, please see:\n"
-> ~"<https://www.gnu.org/software/gdb/bugs/>.\n"
-> ~"Find the GDB manual and other documentation resources online at:\n    <"
-> ~"http://www.gnu.org/software/gdb/documentation/>.\n\n"
-> ~"For help, type \"help\".\n"
-> ~"Type \"apropos word\" to search for commands related to \"word\".\n"
-> 1^done
2-gdb-set mi-async on
-> 2^done
3-interpreter-exec console "set print demangle on"
-> 3^done
4-interpreter-exec console "set print asm-demangle on"
-> =cmd-param-changed,param="print asm-demangle",value="on"
-> 4^done
5-enable-pretty-printing
-> 5^done
6-interpreter-exec console "source /home/marcus/.vscode/extensions/marus25.cortex-debug-1.12.1/support/gdbsupport.init"
-> 6^done
7-interpreter-exec console "source /home/marcus/.vscode/extensions/marus25.cortex-debug-1.12.1/support/gdb-swo.init"
-> =cmd-param-changed,param="language",value="c"
-> =cmd-param-changed,param="language",value="auto"
-> 7^done
8-interpreter-exec console "set output-radix 0xa"
-> ~"Output radix now set to decimal 10, hex a, octal 12.\n"
Output radix now set to decimal 10, hex a, octal 12.
-> 8^done
9-interpreter-exec console "set input-radix 0xa"
-> ~"Input radix now set to decimal 10, hex a, octal 12.\n"
Input radix now set to decimal 10, hex a, octal 12.
-> 9^done
10-file-exec-and-symbols "/home/marcus/src/FreeRTOS/build/sesame.axf"
-> 10^done
Finished reading symbols from nm: Time: 346 ms
11-target-select extended-remote localhost:50000
-> 11^error,msg="Remote replied unexpectedly to 'vMustReplyEmpty': PacketSize=4000;qXfer:memory-map:read-;qXfer:features:read+;qXfer:threads:read+;QStartNoAckMode+;vContSupported+"
12-interpreter-exec console "monitor reset halt"
Failed to launch GDB: Remote replied unexpectedly to 'vMustReplyEmpty': PacketSize=4000;qXfer:memory-map:read-;qXfer:features:read+;qXfer:threads:read+;QStartNoAckMode+;vContSupported+ (from target-select extended-remote localhost:50000)

Additional context

When starting OpenOCD and gdb outside of VS.Code on the command line, I am able to connect to the target.

haneefdm commented 2 days ago

What is going on in the terminal tab where you see the gdb-server output appears?

We only sent 11 commands to gdb before failure. You can see how OpenOCD and GDB are launched and you can launch them the same way and send the same gdb commands and see what happens It appears gdb is not happy with what openocd is sending. We don't talk to openocd at all. Maybe a very old/incomplete version of openocd?

11-target-select extended-remote localhost:50000
-> 11^error,msg="Remote replied unexpectedly to 'vMustReplyEmpty': PacketSize=4000;qXfer:memory-map:read-;qXfer:features:read+;qXfer:threads:read+;QStartNoAckMode+;vContSupported+"

marcusb commented 1 day ago

This is the gdb-server terminal:

Waiting for gdb server to start...[2024-10-22T04:14:01.763Z] SERVER CONSOLE DEBUG: onBackendConnect: gdb-server session connected. You can switch to "DEBUG CONSOLE" to see GDB interactions.
openocd -c "gdb_port 50000" -c "tcl_port 50001" -c "telnet_port 50002" -s /home/marcus/src/FreeRTOS/app/marvell/WMSDK/mw320/sdk/tools/OpenOCD/interface -s /home/marcus/src/FreeRTOS/app/marvell/WMSDK/mw320/sdk/tools/OpenOCD -f /home/marcus/.vscode/extensions/marus25.cortex-debug-1.12.1/support/openocd-helpers.tcl -f /home/marcus/src/FreeRTOS/app/marvell/WMSDK/mw320/sdk/tools/OpenOCD/interface/ftdi.cfg -f /home/marcus/src/FreeRTOS/app/marvell/WMSDK/mw320/sdk/tools/OpenOCD/openocd.cfg -c init -c "load /home/marcus/src/FreeRTOS/build/sesame.axf 0x10013d"
Open On-Chip Debugger 0.12.0
Licensed under GNU GPL v2
For bug reports, read
        http://openocd.org/doc/doxygen/bugs.html
CDLiveWatchSetup
jtag
sh_load
Info : clock speed 2000 kHz
Info : JTAG tap: wmcore.cpu tap/device found: 0x4ba00477 (mfg: 0x23b (ARM Ltd), part: 0xba00, ver: 0x4)
Info : [wmcore.cpu] Cortex-M4 r0p1 processor detected
Info : [wmcore.cpu] target has 6 breakpoints, 4 watchpoints
Info : starting gdb server for wmcore.cpu on 50000
Info : Listening on port 50000 for gdb connections
Info : JTAG tap: wmcore.cpu tap/device found: 0x4ba00477 (mfg: 0x23b (ARM Ltd), part: 0xba00, ver: 0x4)
[wmcore.cpu] halted due to debug-request, current mode: Thread 
xPSR: 0x01000000 pc: 0x00007f14 msp: 0x20001000
DEPRECATED! use 'read_memory' not 'mem2array'
Info : Listening on port 50001 for tcl connections
Info : Listening on port 50002 for telnet connections
Info : accepting 'gdb' connection on tcp/50000
Configuring OS Awareness
Warn : GDB connection 1 on target wmcore.cpu not halted
Error: GDB missing ack(2) - assumed good
Error: GDB missing ack(2) - assumed good
Shutting down
shutdown command invoked
Info : dropped 'gdb' connection
[2024-10-22T04:14:07.878Z] SERVER CONSOLE DEBUG: onBackendConnect: gdb-server session closed
GDB server session ended. This terminal will be reused, waiting for next session to start...

Some googling indicates that it could be caused by sending commands to quickly, or something timing-related like that.

marcusb commented 1 day ago

OpenOCD is the latest version, and it appears to work when sending the commands manually.

Marus / cortex-debug

"Remote replied unexpectedly to 'vMustReplyEmpty'" with OpenOCD #1054