J-Link plugin thread aware debugging (issues on Windows)

[kistlin]

Hello,

when trying to use the J-Link plugin to do thread aware debugging the JLinkGDBServer crashes/shows errors. I compiled the plugin from master and an older version that goes along the sources I use. master: 447a0bce958b9fca11c5505c6b6ebcd2a961d125

To compile, the MSYS2 mingw64 shell is used. I had no luck using the Makefile.host, so I ran it standalone. Original entry

$(Q) $(HOSTCC) $(HOSTCFLAGS) -shared -fPIC jlink-nuttx.c -o jlink-nuttx$(HOSTDYNEXT)

Standalone compilation used

gcc -shared -fPIC jlink-nuttx.c -o jlink-nuttx.dll

SEGGER J-Link version used

SEGGER J-Link GDB Server V7.88 Command Line Version

JLinkARM.dll V7.88 (DLL compiled Apr 28 2023 08:36:41)

NuttX compile option used

CONFIG_DEBUG_TCBINFO=y

Target

STM32H743ZI

Output MSYS2 MINGW64 (older sources)

Loading RTOS plugin: C:/jlink-nuttx.dll...
RTOS plugin (API v1.0) loaded successfully
RTOS plugin: Loaded
Received symbol: g_pidhash (0x240027C8)
Received symbol: g_npidhash (0x240027CC)
Received symbol: g_tcbinfo (0x080CDAB8)
Received symbol: g_readytorun (0x24002788)
All mandatory symbols successfully loaded.
ERROR: error in get_idx_from_pid return -1
Reading common registers: 

MSYS2 MINGW64 (master)

Loading RTOS plugin: C:/jlink-nuttx.dll...
RTOS plugin (API v1.1) loaded successfully
RTOS plugin: Loaded
Received symbol: g_pidhash (0x240027C8)
Received symbol: g_npidhash (0x240027CC)
Received symbol: g_tcbinfo (0x080CDAB8)
Received symbol: g_readytorun (0x24002788)
All mandatory symbols successfully loaded.
WARNING: Failed to read memory @ address 0x010000FC
ERROR: error in read tcbinfo_s reg_offs ret 0
npidhash change from 0 to 128!
ERROR: error in get_idx_from_pid return -2

Just to test a mingw32 build. But it doesn't match the SEGGER 64-bit tooling.

MSYS2 MINGW32
Loading RTOS plugin: C:/jlink-nuttx.dll...
RTOS plugin load failed (Error 193, %1 is not a valid Win32 application.).

[acassis]

Hi @kistlin I already used JLinkGDBServer to debug NuttX but not with system awareness: https://acassis.wordpress.com/2015/06/29/debugging-nuttx-on-atmel-sam-l21-microcontroller-with-j-link-programmer/

Looking your error message I can see that it is failing to get the idx of pid "ERROR: error in get_idx_from_pid return -1" and the only way to get_idx_from_pid() return -1 is if "get_pid()" fails.

@xiaoxiang781216 do you know if there is some recent modification on tcbinfo that could have broken JLink awareness?

I was trying to find a tutorial/guide about JLink aware on our documentation https://nuttx.apache.org/docs/latest/guides/index.html but just noticed it doesn't exist. @kistlin could you help us to fix it?

Update: maybe we need to expand the documentation here: https://nuttx.apache.org/docs/10.0.0/quickstart/debugging.html#jtag-debugging

[kistlin]

Yes debugging without system awareness works fine for me too.

And the older sources of jlink-nuttx.c I mentioned, are from the first commit 407bc04934737cdf9e473a64b4663c6aefa9312a. The NuttX sources are also from that time period. So I would assume that the tcbinfo most likely was similar at that time.

Yes I can help. No problem.

[kistlin]

And by the way I just saw that SEGGER is shipping their own binaries for RTOSes. In C:\Program Files\SEGGER\JLink\GDBServer there is

RTOSPlugin_Azure.dll
RTOSPlugin_ChibiOS.dll
RTOSPlugin_embOS.dll
RTOSPlugin_FreeRTOS.dll
RTOSPlugin_NuttX.dll
RTOSPlugin_Zephyr.dll

and probably similar for Linux/macOS builds.

When running it doesn't throw errors, but doesn't work :(. Debugging just stops after reading registers, flashing the binary and reading some bytes.

Loading RTOS plugin: C:/Program Files/SEGGER/JLink/GDBServer/RTOSPlugin_NuttX.dll...
RTOS plugin (API v1.2) loaded successfully
RTOS plugin: Loaded
Received symbol: g_running_tasks (0x24002790)
Received symbol: g_readytorun (0x24002788)
Received symbol: g_pendingtasks (0x24002794)
Received symbol: g_waitingforsemaphore (0x2400279C)
Received symbol: g_waitingforsignal (0x240027A4)
All mandatory symbols successfully loaded.

Maybe there is something else not good.

[kistlin]

Ok maybe it is the Visual Studio Code plugin. I now used CLion and configured it to run JLinkGDBServerCL.exe with additional flag -rtos GDBServer/RTOSPlugin_NuttX.dll. As mentioned in J-Link / J-Trace User Guide 4.5.21 -rtos. There I now see a thread overview, and which thread is hitting the breakpoint.

[xiaoxiang781216]

Hi @kistlin I already used JLinkGDBServer to debug NuttX but not with system awareness: https://acassis.wordpress.com/2015/06/29/debugging-nuttx-on-atmel-sam-l21-microcontroller-with-j-link-programmer/

Looking your error message I can see that it is failing to get the idx of pid "ERROR: error in get_idx_from_pid return -1" and the only way to get_idx_from_pid() return -1 is if "get_pid()" fails.

@xiaoxiang781216 do you know if there is some recent modification on tcbinfo that could have broken JLink awareness?

We just use the plugin in Linux environment. @anjiahao1 could you try the plugin?

[kistlin]

Ok. I prefer to work with WSL2 and my setup is number two in the image of this comment https://github.com/Marus/cortex-debug/issues/795#issue-1510793349 in this case. That's why I need the Windows version of the plugin.

[acassis]

@xiaoxiang781216 I thought you had submitted some documentation about JLink NuttX awaress but I didn't find it. If you or @kistlin describe the steps I can duplicate it here I can write the documentation.

I ran: $ make -f Makefile.host jlink-nuttx and it generated the .so library (@kistlin please try this command on Windows), I think I need to replace my /opt/SEGGER/JLink_V780c/GDBServer/RTOSPlugin_NuttX.so with this generated .so. Correct?

I used readelf to dump info from each .so, first from Segger: https://pastebin.com/raw/0my7sSSF

And then from my generated jlink-nuttx.so: https://pastebin.com/raw/Hd3tbZN3

They have many differences. For example our is missing SONAME: "Library soname: [RTOSPlugin_NuttX.so]"

Guys, we do think we need to pass some the "-soname" when generating the libray and also use same RTOSPlugin_NuttX.so to make it clear that the user will need to replace it on JLink GDB Server?

[kistlin]

@acassis I'm on Windows and I used MSYS2 mingw64 as mentioned above, due to my setup. Then to compile the library use

gcc -shared -fPIC jlink-nuttx.c -o jlink-nuttx.dll

I copied that out of the Makefile.host script.

When running JLinkGDBServerCL.exe, next to the other arguments you can pass -rtos GDBServer/RTOSPlugin_NuttX.dll. It also takes absolute paths, so you don't need to name it the same or replace it. That should already be it.

[anchao]

I just tried another stm32 board on Linux(Ubuntu 22.04), thread aware debugging works as expected:

1. make image:

   
   ./tools/configure.sh stm32f429i-disco:nsh

enable CONFIG_DEBUG_TCBINFO=y CONFIG_TASK_NAME_SIZE=32

2. connect the jlink:

$ make -f Makefile.host jlink-nuttx.so $ JLinkGDBServer -if SWD -device Cortex-M33 -rtos $PWD/jlink-nuttx.so SEGGER J-Link GDB Server V7.88a Command Line Version

JLinkARM.dll V7.88a (DLL compiled May 3 2023 14:37:20)

Command line: -if SWD -device Cortex-M33 -rtos /home/archer/code/nuttx/n11/incubator-nuttx/tools/jlink-nuttx.so -----GDB Server start settings----- GDBInit file: none GDB Server Listening port: 2331 SWO raw output listening port: 2332 Terminal I/O port: 2333 Accept remote connection: yes Generate logfile: off Verify download: off Init regs on start: off Silent mode: off Single run mode: off Target connection timeout: 0 ms ------J-Link related settings------ J-Link Host interface: USB J-Link script: none J-Link settings file: none ------Target related settings------ Target device: Cortex-M33 Target device parameters: none Target interface: SWD Target interface speed: 4000kHz Target endian: little

Connecting to J-Link... J-Link is connected. Firmware: J-Link V11 compiled Mar 28 2023 16:59:55 Hardware: V11.00 S/N: 51015399 Feature(s): GDB Checking target voltage... Target voltage: 2.89 V Listening on TCP/IP port 2331 Connecting to target... Halting core... Connected to target Waiting for GDB connection...Connected to 127.0.0.1 GDB client (conn. 9) requested target.xml from GDB Server Reading common registers: Read register 'r0' (4 bytes) from hardware: 0x00000000 Read register 'r1' (4 bytes) from hardware: 0x0C040020 Read register 'r2' (4 bytes) from hardware: 0xDC0C0010 Read register 'r3' (4 bytes) from hardware: 0x00000000 Read register 'r4' (4 bytes) from hardware: 0x00000000 Read register 'r5' (4 bytes) from hardware: 0x00000000 Read register 'r6' (4 bytes) from hardware: 0x00000000 Read register 'r7' (4 bytes) from hardware: 0x00000000 Read register 'r8' (4 bytes) from hardware: 0x00000000 Read register 'r9' (4 bytes) from hardware: 0x00000000 Read register 'r10' (4 bytes) from hardware: 0x00000000 Read register 'r11' (4 bytes) from hardware: 0x00000000 Read register 'r12' (4 bytes) from hardware: 0x00000000 Read register 'sp' (4 bytes) from hardware: 0x840D0020 Read register 'lr' (4 bytes) from hardware: 0xC91A0008 Read register 'pc' (4 bytes) from hardware: 0xC81A0008 Read register 'xpsr' (4 bytes) from hardware: 0x00000041 Read 4 bytes @ address 0x08001AC8 (Data = 0xBF00E7FC) Loading RTOS plugin: //home/archer/code/nuttx/n11/incubator-nuttx/tools/jlink-nuttx.so... RTOS plugin (API v1.1) loaded successfully RTOS plugin: Loaded Received symbol: g_pidhash (0x20000380) Received symbol: g_npidhash (0x20000384) Received symbol: g_tcbinfo (0x0801D7C8) Received symbol: g_readytorun (0x20000358) All mandatory symbols successfully loaded. npidhash change from 0 to 4! ERROR: error in get_idx_from_pid return -2 Read 4 bytes @ address 0x08001AC8 (Data = 0xBF00E7FC) Reading common registers: Read register 'r0' (4 bytes) from hardware: 0x00000000 Read register 'r1' (4 bytes) from hardware: 0x0C040020 Read register 'r2' (4 bytes) from hardware: 0xDC0C0010 Read register 'r3' (4 bytes) from hardware: 0x00000000 Read register 'r4' (4 bytes) from hardware: 0x00000000 Read register 'r5' (4 bytes) from hardware: 0x00000000 Read register 'r6' (4 bytes) from hardware: 0x00000000 Read register 'r7' (4 bytes) from hardware: 0x00000000 Read register 'r8' (4 bytes) from hardware: 0x00000000

$ arm-none-eabi-gdb ./nuttx -ex "target remote localhost:2331" GNU gdb (Arm GNU Toolchain 11.3.Rel1) 12.1.90.20220802-git Copyright (C) 2022 Free Software Foundation, Inc. License GPLv3+: GNU GPL version 3 or later http://gnu.org/licenses/gpl.html This is free software: you are free to change and redistribute it. There is NO WARRANTY, to the extent permitted by law. Type "show copying" and "show warranty" for details. This GDB was configured as "--host=x86_64-pc-linux-gnu --target=arm-none-eabi". Type "show configuration" for configuration details. For bug reporting instructions, please see: https://bugs.linaro.org/. Find the GDB manual and other documentation resources online at: http://www.gnu.org/software/gdb/documentation/.

For help, type "help". Type "apropos word" to search for commands related to "word"... Reading symbols from ./nuttx... Remote debugging using localhost:2331 up_idle () at chip/stm32_idle.c:195 195 } (gdb) info thread [New Thread 1] [New Thread 2] [New Thread 3] Id Target Id Frame

• 1 Thread 57005 ([PID:000]Idle Task:0003[PRI:000]) up_idle () at chip/stm32_idle.c:195 2 Thread 1 ([PID:000]Idle Task:0003[PRI:000]) up_idle () at chip/stm32_idle.c:195 3 Thread 2 ([PID:001]nsh_main:0005[PRI:100]) sys_call2 (nbr=2, parm1=268435956, parm2=268442532) at /home/archer/code/nuttx/n11/incubator-nuttx/include/arch/syscall.h:187 4 Thread 3 ([PID:002]hello:0006[PRI:100]) sys_call2 (nbr=2, parm1=268439708, parm2=536874324) at /home/archer/code/nuttx/n11/incubator-nuttx/include/arch/syscall.h:187 (gdb) thread 4 [Switching to thread 4 (Thread 3)]

  0 sys_call2 (nbr=2, parm1=268439708, parm2=536874324) at /home/archer/code/nuttx/n11/incubator-nuttx/include/arch/syscall.h:187

  187 return reg0; (gdb) thread 1 Thread ID 1 has terminated. (gdb) bt

  0 sys_call2 (nbr=2, parm1=268439708, parm2=536874324) at /home/archer/code/nuttx/n11/incubator-nuttx/include/arch/syscall.h:187

  1 0x0800a7a8 in up_switch_context (tcb=0x2000038c , rtcb=0x10001020) at common/arm_switchcontext.c:95

  2 0x0801828a in nxsig_timedwait (set=0x10001b24, info=0x0, timeout=0x10001b98) at signal/sig_timedwait.c:367

  3 0x08017f4e in nxsig_nanosleep (rqtp=0x10001b98, rmtp=0x10001b90) at signal/sig_nanosleep.c:123

  4 0x0801806c in clock_nanosleep (clockid=0 '\000', flags=0, rqtp=0x10001b98, rmtp=0x10001b90) at signal/sig_nanosleep.c:308

  5 0x080198e0 in sleep (seconds=1) at unistd/lib_sleep.c:110

  6 0x0801a8e2 in hello_main (argc=1, argv=0x10001408) at hello_main.c:41

  7 0x080080be in nxtask_startup (entrypt=0x801a8c9 , argc=1, argv=0x10001408) at sched/task_startup.c:70

  8 0x08003906 in nxtask_start () at task/task_start.c:134

  9 0x00000000 in ?? ()

  (gdb) thread 2 [Switching to thread 2 (Thread 1)]

  0 up_idle () at chip/stm32_idle.c:195

  195 } (gdb) bt

  0 up_idle () at chip/stm32_idle.c:195

  1 0x08001ac4 in nx_start () at init/nx_start.c:698

  2 0x08000202 in __start () at chip/stm32_start.c:199

  (gdb) thread 3 [Switching to thread 3 (Thread 2)]

  0 sys_call2 (nbr=2, parm1=268435956, parm2=268442532) at /home/archer/code/nuttx/n11/incubator-nuttx/include/arch/syscall.h:187

  187 return reg0; (gdb) bt

  0 sys_call2 (nbr=2, parm1=268435956, parm2=268442532) at /home/archer/code/nuttx/n11/incubator-nuttx/include/arch/syscall.h:187

  1 0x0800a7a8 in up_switch_context (tcb=0x10001020, rtcb=0x10000178) at common/arm_switchcontext.c:95

  2 0x08004bd4 in nxsem_wait (sem=0x20000020 <g_usart1priv+32>) at semaphore/sem_wait.c:176

  3 0x080070cc in uart_read (filep=0x100004c0, buffer=0x10000c53 "\b\244\f", buflen=1) at serial/serial.c:1063

  4 0x08013126 in file_read (filep=0x100004c0, buf=0x10000c53, nbytes=1) at vfs/fs_read.c:92

  5 0x0801315e in nx_read (fd=0, buf=0x10000c53, nbytes=1) at vfs/fs_read.c:140

  6 0x0801317c in read (fd=0, buf=0x10000c53, nbytes=1) at vfs/fs_read.c:170

  7 0x0800af0a in readline_getc (vtbl=0x10000ca4) at readline_fd.c:72

  8 0x0800b042 in readline_common (vtbl=0x10000ca4, buf=0x10000fd8 "hello", buflen=64) at readline_common.c:513

  9 0x0800aff4 in readline_fd (buf=0x10000fd8 "hello", buflen=64, infd=0, outfd=1) at readline_fd.c:222

  10 0x0800ae80 in nsh_session (pstate=0x10000d50, login=1, argc=1, argv=0x10000560) at nsh_session.c:212

  11 0x0800aad8 in nsh_consolemain (argc=1, argv=0x10000560) at nsh_consolemain.c:71

  12 0x0800aa9a in nsh_main (argc=1, argv=0x10000560) at nsh_main.c:74

  13 0x080080be in nxtask_startup (entrypt=0x800aa65 , argc=1, argv=0x10000560) at sched/task_startup.c:70

  14 0x08003906 in nxtask_start () at task/task_start.c:134

  15 0x00000000 in ?? ()

[kistlin]

@anchao thanks for testing. The only difference is I used a dll for Windows.

But the command line somewhat works.

For VS Code there is already an open issue https://github.com/microsoft/vscode-cpptools/issues/10848.

Startup of the GDB server

Loading RTOS plugin: C:/../jlink-nuttx.dll...
RTOS plugin (API v1.1) loaded successfully
RTOS plugin: Loaded
Received symbol: g_pidhash (0x240029A8)
Received symbol: g_npidhash (0x240029AC)
Received symbol: g_tcbinfo (0x080D74B8)
Received symbol: g_readytorun (0x24002968)
All mandatory symbols successfully loaded.
WARNING: Failed to read memory @ address 0x010000FC
ERROR: error in read tcbinfo_s reg_offs ret 0
npidhash change from 0 to 128!
ERROR: error in get_idx_from_pid return -2

and I get lots of

Reading common registers: ERROR: regread 1 regidx 0 error 1
ERROR: Internal error: Register 'r0': RTOS plugin reports 0 bytes register size, while GDB Server expects 4 bytes
Read register 'r0' (4 bytes) from hardware: 0x00000000
ERROR: regread 1 regidx 1 error 1
ERROR: Internal error: Register 'r1': RTOS plugin reports 0 bytes register size, while GDB Server expects 4 bytes
Read register 'r1' (4 bytes) from hardware: 0x402A0024
ERROR: regread 1 regidx 2 error 1

Here is a command line example output of info thread

  Id   Target Id                                                           Frame
* 1    Thread 57005 ()                                                    0x0800a8cc in up_idle () at common/arm_idle.c:63
  2    Thread 1 ([PID:000]Idle Task:0003[PRI:000])                        0x0800a8cc in up_idle () at common/arm_idle.c:63
  3    Thread 3772776450 ([PID:-522190847]<name...>:0005[PRI:224])        0x0800a8cc in up_idle () at common/arm_idle.c:63
  4    Thread 1684275203 ([PID:1684275202]<name...>:0005[PRI:100])        0x0800a8cc in up_idle () at common/arm_idle.c:63
  5    Thread 1684275204 ([PID:1684275203]<name...>:0006[PRI:100])        0x0800a8cc in up_idle () at common/arm_idle.c:63
  6    Thread 1684275205 ([PID:1684275204]<name...>:0006[PRI:100])        0x0800a8cc in up_idle () at common/arm_idle.c:63
  7    Thread 1684275206 ([PID:1684275205]<name...>:0006[PRI:100])        0x0800a8cc in up_idle () at common/arm_idle.c:63
  8    Thread 1515847687 ([PID:1515847686]<name...>:0006[PRI:090])        0x0800a8cc in up_idle () at common/arm_idle.c:63
  9    Thread 1684275208 ([PID:1684275207]<name...>:0007[PRI:100])        0x0800a8cc in up_idle () at common/arm_idle.c:63
  10   Thread 1684275209 ([PID:1684275208]<name...>:0005[PRI:100])        0x0800a8cc in up_idle () at common/arm_idle.c:63
  11   Thread 3368550410 ([PID:-926416887]<name...>:0007[PRI:200])        0x0800a8cc in up_idle () at common/arm_idle.c:63
  12   Thread 2610626571 ([PID:-1684340726]<name...>:0005[PRI:155])       0x0800a8cc in up_idle () at common/arm_idle.c:63
  13   Thread 2947481612 ([PID:-1347485685]<name...>:0007[PRI:175])       0x0800a8cc in up_idle () at common/arm_idle.c:63
  14   Thread 2947481613 ([PID:-1347485684]<name...>:0007[PRI:175])       0x0800a8cc in up_idle () at common/arm_idle.c:63
  15   Thread 842137614 ([PID:842137613]<name...>:0006[PRI:050])          0x0800a8cc in up_idle () at common/arm_idle.c:63
  16   Thread 1684275215 ([PID:1684275214]<name...>:0006[PRI:100])        0xbeefdeac in ?? ()
  17   Thread 1684275216 ([PID:1684275215]<name...>:0007[PRI:100])        0x0800a8cc in up_idle () at common/arm_idle.c:63
  18   Thread 1684275217 ([PID:1684275216]<name...>:0006[PRI:100])        0xbeefdeac in ?? ()
  19   Thread 1684275218 ([PID:1684275217]<name...>:0006[PRI:100])        0x00000000 in ?? ()
  20   Thread 1684275219 ([PID:1684275218]<name...>:0007[PRI:100])        0x0800a8cc in up_idle () at common/arm_idle.c:63
  21   Thread 1684275220 ([PID:1684275219]<name...>:0007[PRI:100])        0x0800a8cc in up_idle () at common/arm_idle.c:63
  22   Thread 1684275221 ([PID:1684275220]<name...>:0006[PRI:100])        0x0800a8cc in up_idle () at common/arm_idle.c:63
  23   Thread 1684275222 ([PID:1684275221]<name...>:0007[PRI:100])        0x0800a8cc in up_idle () at common/arm_idle.c:63
  24   Thread 1684275223 ([PID:1684275222]<name...>:0006[PRI:100])        0x0800a8cc in up_idle () at common/arm_idle.c:63
  25   Thread 1684275224 ([PID:1684275223]<name...>:0007[PRI:100])        0x0800a8cc in up_idle () at common/arm_idle.c:63
  26   Thread 1684275225 ([PID:1684275224]<name...>:0005[PRI:100])        0x0800a8cc in up_idle () at common/arm_idle.c:63
  27   Thread 2947481626 ([PID:-1347485671]<name...>:0007[PRI:175])       0x0800a8cc in up_idle () at common/arm_idle.c:63
  28   Thread 1684275227 ([PID:1684275226]<name...>:0005[PRI:100])        0x0800a8cc in up_idle () at common/arm_idle.c:63
  29   Thread 1684275228 ([PID:1684275227]<name...>:0007[PRI:100])        0x0800a8cc in up_idle () at common/arm_idle.c:63
  30   Thread 1684275229 ([PID:1684275228]<name...>:0007[PRI:100])        0x0800a8cc in up_idle () at common/arm_idle.c:63
  31   Thread 1667432478 ([PID:1667432477]<name...>:0006[PRI:099])        0x00000000 in ?? ()
  32   Thread 1684275231 ([PID:1684275230]<name...>:0005[PRI:100])        0x0800a8cc in up_idle () at common/arm_idle.c:63
  33   Thread 1684275232 ([PID:1684275231]<name...>:0007[PRI:100])        0x0800a8cc in up_idle () at common/arm_idle.c:63
  34   Thread 1684275233 ([PID:1684275232]<name...>:0006[PRI:100])        0x0800a8cc in up_idle () at common/arm_idle.c:63
  35   Thread 2526412834 ([PID:-1768554463]<name...>:0007[PRI:150])       0x0800a8cc in up_idle () at common/arm_idle.c:63
  36   Thread 1684275235 ([PID:1684275234]<name...>:0007[PRI:100])        0x0800a8cc in up_idle () at common/arm_idle.c:63
  37   Thread 1684275236 ([PID:1684275235]<name...>:0007[PRI:100])        0x0800a8cc in up_idle () at common/arm_idle.c:63
  38   Thread 1684275237 ([PID:1684275236]<name...>:0007[PRI:100])        0x00000000 in ?? ()
  39   Thread 1684275238 ([PID:1684275237]<name...>:0007[PRI:100])        0x0800a8cc in up_idle () at common/arm_idle.c:63
  40   Thread 1684275239 ([PID:1684275238]<name...>:0007[PRI:100])        0x0800a8cc in up_idle () at common/arm_idle.c:63
  41   Thread 1684275240 ([PID:1684275239]<name...>:0007[PRI:100])        0x0800a8cc in up_idle () at common/arm_idle.c:63
  42   Thread 1684275241 ([PID:1684275240]<name...>:0006[PRI:100])        0x0800a8cc in up_idle () at common/arm_idle.c:63
  43   Thread 2947481642 ([PID:-1347485655]<name...>:0007[PRI:175])       0x0800a8cc in up_idle () at common/arm_idle.c:63
  44   Thread 1684275243 ([PID:1684275242]<name...>:0006[PRI:100])        0x00000000 in ?? ()
  45   Thread 2947481644 ([PID:-1347485653]<name...>:0007[PRI:175])       0x0800a8cc in up_idle () at common/arm_idle.c:63
  46   Thread 1684275245 ([PID:1684275244]<name...>:0007[PRI:100])        0x0800a8cc in up_idle () at common/arm_idle.c:63
  47   Thread 1684275246 ([PID:1684275245]<name...>:0007[PRI:100])        0x0800a8cc in up_idle () at common/arm_idle.c:63
  48   Thread 1684275247 ([PID:1684275246]<name...>:0007[PRI:100])        0x0800a8cc in up_idle () at common/arm_idle.c:63
  49   Thread 1852702768 ([PID:1852702767]<name...>:0006[PRI:110])        0x00000000 in ?? ()
  50   Thread 1684275249 ([PID:1684275248]<name...>:0007[PRI:100])        0xbeefdeac in ?? ()
  51   Thread 1684275250 ([PID:1684275249]<name...>:0007[PRI:100])        0x0800a8cc in up_idle () at common/arm_idle.c:63
  52   Thread 1684275251 ([PID:1684275250]<name...>:0005[PRI:100])        0x0800a8cc in up_idle () at common/arm_idle.c:63
  53   Thread 1684275252 ([PID:1684275251]<name...>:0007[PRI:100])        0x0800a8cc in up_idle () at common/arm_idle.c:63
  54   Thread 1684275253 ([PID:1684275252]<name...>:0006[PRI:100])        0x0800a8cc in up_idle () at common/arm_idle.c:63
  55   Thread 1684275254 ([PID:1684275253]<name...>:0005[PRI:100])        0x0800a8cc in up_idle () at common/arm_idle.c:63
  56   Thread 1684275255 ([PID:1684275254]<name...>:0007[PRI:100])        0x0800a8cc in up_idle () at common/arm_idle.c:63
  57   Thread 1684275256 ([PID:1684275255]<name...>:0005[PRI:100])        0x0800a8cc in up_idle () at common/arm_idle.c:63
  58   Thread 842137657 ([PID:842137656]<name...>:0005[PRI:050])          0xf9907f9a in ?? ()
  59   Thread 842137658 ([PID:842137657]<name...>:0005[PRI:050])          0x0800a8cc in up_idle () at common/arm_idle.c:63
  60   Thread 842137659 ([PID:842137658]<name...>:0006[PRI:050])          0x1cbd9bfe in ?? ()
  61   Thread 1684275260 ([PID:1684275259]<name...>:0007[PRI:100])        0x0800a8cc in up_idle () at common/arm_idle.c:63
  62   Thread 1852702781 ([PID:1852702780]<name...>:0007[PRI:110])        0x0800a8cc in up_idle () at common/arm_idle.c:63
  63   Thread 1852702782 ([PID:1852702781]<name...>:0006[PRI:110])        0xbeefdeac in ?? ()
  64   Thread 1010565183 ([PID:1010565182]<name...>:0007[PRI:060])        0x0800a8cc in up_idle () at common/arm_idle.c:63
  65   Thread 1684275264 ([PID:1684275263]<name...>:0007[PRI:100])        0x0800a8cc in up_idle () at common/arm_idle.c:63
  66   Thread 757923905 ([PID:757923904]<name...>:0005[PRI:045])          0x0800a8cc in up_idle () at common/arm_idle.c:63
  67   Thread 842137666 ([PID:842137665]<name...>:0006[PRI:050])          0x0800a8cc in up_idle () at common/arm_idle.c:63
  68   Thread 1852702787 ([PID:1852702786]<name...>:0006[PRI:110])        0x0800a8cc in up_idle () at common/arm_idle.c:63
  69   Thread 842137668 ([PID:842137667]<name...>:0006[PRI:050])          0x0800a8cc in up_idle () at common/arm_idle.c:63
  70   Thread 842137669 ([PID:842137668]<name...>:0006[PRI:050])          0x00000000 in ?? ()
  71   Thread 1684275270 ([PID:1684275269]<name...>:0007[PRI:100])        0x0800a8cc in up_idle () at common/arm_idle.c:63
  72   Thread 1852702791 ([PID:1852702790]<name...>:0006[PRI:110])        0x0800a8cc in up_idle () at common/arm_idle.c:63
  73   Thread 1852702792 ([PID:1852702791]<name...>:0006[PRI:110])        0xbeefdeac in ?? ()

It looks ok but I'm not 100% convinced yet, if all these errors are normal.

[anjiahao1]

@kistlin hi,I often use linux, not very familiar with windows, can you tell me how to use the command line jlinkGdbServers to load the dll library on win

[anjiahao1]

i use .\JLinkGDBServer.exe -rtos jlink-nuttx.dll -if SWD -speed 4000 -device STM32F429ZI but jlink log is

SEGGER J-Link GDB Server V7.84b GUI Version

JLinkARM.dll V7.84b (DLL compiled Jan 11 2023 16:42:23)

-----GDB Server start settings-----
GDBInit file:                  none
GDB Server Listening port:     2331
SWO raw output listening port: 2332
Terminal I/O port:             2333
Accept remote connection:      localhost only
Generate logfile:              off
Verify download:               off
Init regs on start:            off
Silent mode:                   off
Single run mode:               off
Target connection timeout:     5000 ms
------J-Link related settings------
J-Link Host interface:         USB
J-Link script:                 none
J-Link settings file:          none
------Target related settings------
Target device:                 STM32F429ZI
Target device parameters:      none
Target interface:              SWD
Target interface speed:        4000kHz
Target endian:                 little

Connecting to J-Link...
J-Link is connected.
Firmware: J-Link V11 compiled Mar 28 2023 16:59:55
Hardware: V11.00
S/N: 51015399
Feature(s): GDB
Checking target voltage...
Target voltage: 2.83 V
Listening on TCP/IP port 2331
Connecting to target...
Connected to target
Waiting for GDB connection...

not have load dll file

[anjiahao1]

$ ./JLinkGDBServerCL.exe -rtos ~/work/nuttx/jlink-nuttx.dll -if SWD -speed 4000 -device STM32F429ZI
SEGGER J-Link GDB Server V7.84b Command Line Version

JLinkARM.dll V7.84b (DLL compiled Jan 11 2023 16:42:23)

Command line: -rtos C:/msys64/home/ajh11/work/nuttx/jlink-nuttx.dll -if SWD -speed 4000 -device STM32F429ZI
-----GDB Server start settings-----
GDBInit file:                  none
GDB Server Listening port:     2331
SWO raw output listening port: 2332
Terminal I/O port:             2333
Accept remote connection:      localhost only
Generate logfile:              off
Verify download:               off
Init regs on start:            off
Silent mode:                   off
Single run mode:               off
Target connection timeout:     0 ms
------J-Link related settings------
J-Link Host interface:         USB
J-Link script:                 none
J-Link settings file:          none
------Target related settings------
Target device:                 STM32F429ZI
Target device parameters:      none
Target interface:              SWD
Target interface speed:        4000kHz
Target endian:                 little

Connecting to J-Link...
J-Link is connected.
Firmware: J-Link V11 compiled Mar 28 2023 16:59:55
Hardware: V11.00
S/N: 51015399
Feature(s): GDB
Checking target voltage...
Target voltage: 2.94 V
Listening on TCP/IP port 2331
Connecting to target...
Connected to target
Waiting for GDB connection...Connected to 127.0.0.1
Reading common registers: R0, R1, R2, R3, R4, R5, R6, R7, R8, R9, R10, R11, R12, SP, LR, PC, XPSR
Read 4 bytes @ address 0x08001AEA (Data = 0xFB69F008)
Reading 64 bytes @ address 0x20000DC0
Loading RTOS plugin: C:/msys64/home/ajh11/work/nuttx/jlink-nuttx.dll...
RTOS plugin (API v1.1) loaded successfully
RTOS plugin: Loaded
Received symbol: g_pidhash (0x2000038C)
Received symbol: g_npidhash (0x20000390)
Received symbol: g_tcbinfo (0x0801E17C)
Received symbol: g_readytorun (0x20000364)
All mandatory symbols successfully loaded.
Read 4 bytes @ address 0x08000202 (Data = 0x0160BF00)
Read 4 bytes @ address 0xFFFFFFFE (Data = 0x0DDC0000)
Read 2 bytes @ address 0xFFFFFFFE (Data = 0x0000)
npidhash change from 0 to 4!
ERROR: error in get_idx_from_pid return -2
Segmentation fault

i use win get this 🐛

 .\arm-none-eabi-gdb.exe nuttx -ex "target remote 127.0.0.1:2331"
C:\Users\ajh11\Downloads\gcc-arm-none-eabi-10.3-2021.10-win32\gcc-arm-none-eabi-10.3-2021.10\bin\arm-none-eabi-gdb.exe: warning: Couldn't determine a path for the index cache directory.
GNU gdb (GNU Arm Embedded Toolchain 10.3-2021.10) 10.2.90.20210621-git
Copyright (C) 2021 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
Type "show copying" and "show warranty" for details.
This GDB was configured as "--host=i686-w64-mingw32 --target=arm-none-eabi".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
<https://www.gnu.org/software/gdb/bugs/>.
Find the GDB manual and other documentation resources online at:
    <http://www.gnu.org/software/gdb/documentation/>.

For help, type "help".
Type "apropos word" to search for commands related to "word"...
Reading symbols from nuttx...
Remote debugging using 127.0.0.1:2331
nx_start () at init/nx_start.c:697
697     init/nx_start.c: No such file or directory.
(gdb) info threads
[New Thread 1]
[New Thread 2]
Remote communication error.  Target disconnected.: No such file or directory.
(gdb) quit
PS C:\Users\ajh11\Downloads\gcc-arm-none-eabi-10.3-2021.10-win32\gcc-arm-none-eabi-10.3-2021.10\bin> .\arm-none-eabi-gdb.exe nuttx -ex "target remote 127.0.0.1:2331"
C:\Users\ajh11\Downloads\gcc-arm-none-eabi-10.3-2021.10-win32\gcc-arm-none-eabi-10.3-2021.10\bin\arm-none-eabi-gdb.exe: warning: Couldn't determine a path for the index cache directory.
GNU gdb (GNU Arm Embedded Toolchain 10.3-2021.10) 10.2.90.20210621-git
Copyright (C) 2021 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
Type "show copying" and "show warranty" for details.
This GDB was configured as "--host=i686-w64-mingw32 --target=arm-none-eabi".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
<https://www.gnu.org/software/gdb/bugs/>.
Find the GDB manual and other documentation resources online at:
    <http://www.gnu.org/software/gdb/documentation/>.

For help, type "help".
Type "apropos word" to search for commands related to "word"...
Reading symbols from nuttx...
Remote debugging using 127.0.0.1:2331
nx_start () at init/nx_start.c:697
697     init/nx_start.c: No such file or directory.
(gdb) bt
#0  nx_start () at init/nx_start.c:697
#1  0x08000202 in __start () at chip/stm32_start.c:198
(gdb) info threads
[New Thread 1]
[New Thread 2]
Remote communication error.  Target disconnected.: No error.
(gdb)

[anjiahao1]

can you try ubuntu22.04 or wsl? @kistlin

[kistlin]

@anjiahao1 just to be clear.

My setup is running the GDB server on Windows native, to not deal with USB and WSL2. The GDB client runs in WSL2 (Ubuntu 22.04) and connects to the GDB server.

I now tested the plugin on a native Linux. I see almost the same output as in the previous comment, when using the plugin on Windows. But for example no ERROR: error in get_idx_from_pid return -2. Fetching thread information on the command line seems to work.

Loading RTOS plugin: /home/user/nuttx-project/nuttx/tools/jlink-nuttx.so...
RTOS plugin (API v1.1) loaded successfully
RTOS plugin: Loaded
Received symbol: g_pidhash (0x24002968)
Received symbol: g_npidhash (0x2400296C)
Received symbol: g_tcbinfo (0x080D2BE0)
Received symbol: g_readytorun (0x24002928)
All mandatory symbols successfully loaded.
Starting target CPU...
Debugger requested to halt target...
...Target halted (PC = 0x0809EA92)
WARNING: Failed to read memory @ address 0x010000FC
ERROR: error in read tcbinfo_s reg_offs ret 0
npidhash change from 0 to 128!
Reading common registers: Read register 'r0' (4 bytes) from hardware: 0x40130024
Read register 'r1' (4 bytes) from hardware: 0x14000000
Read register 'r2' (4 bytes) from hardware: 0xFBFFFFFF
Read register 'r3' (4 bytes) from hardware: 0x00000000
Read register 'r4' (4 bytes) from hardware: 0x65000000
Read register 'r5' (4 bytes) from hardware: 0x00000000
Read register 'r6' (4 bytes) from hardware: 0xC0400030
Read register 'r7' (4 bytes) from hardware: 0x30BF0030
Read register 'r8' (4 bytes) from hardware: 0x01000000
Read register 'r9' (4 bytes) from hardware: 0x00000000
Read register 'r10' (4 bytes) from hardware: 0xE2380000
Read register 'r11' (4 bytes) from hardware: 0x00000000
Read register 'r12' (4 bytes) from hardware: 0x4A000000
Read register 'sp' (4 bytes) from hardware: 0x30BF0030
Read register 'lr' (4 bytes) from hardware: 0x2FF00908
Read register 'pc' (4 bytes) from hardware: 0x92EA0908
Read register 'xpsr' (4 bytes) from hardware: 0x00000021
Read 4 bytes @ address 0x0809EA92 (Data = 0xB083B480)
Read 4 bytes @ address 0x0809F02E (Data = 0x6C3B4602)
Reading 64 bytes @ address 0x3000BF80
Reading 64 bytes @ address 0x3000BF00
WARNING: Failed to read memory @ address 0x30047F1C
ERROR: error in read tcb name return 0
Reading common registers: ERROR: regread 0 regidx 0 error 1
ERROR: Internal error: Register 'r0': RTOS plugin reports 0 bytes register size, while GDB Server expects 4 bytes
Read register 'r0' (4 bytes) from hardware: 0x40130024
ERROR: regread 0 regidx 1 error 1
ERROR: Internal error: Register 'r1': RTOS plugin reports 0 bytes register size, while GDB Server expects 4 bytes
Read register 'r1' (4 bytes) from hardware: 0x14000000
ERROR: regread 0 regidx 2 error 1
ERROR: Internal error: Register 'r2': RTOS plugin reports 0 bytes register size, while GDB Server expects 4 bytes
Read register 'r2' (4 bytes) from hardware: 0xFBFFFFFF
ERROR: regread 0 regidx 3 error 1
ERROR: Internal error: Register 'r3': RTOS plugin reports 0 bytes register size, while GDB Server expects 4 bytes
Read register 'r3' (4 bytes) from hardware: 0x00000000
ERROR: regread 0 regidx 4 error 1
ERROR: Internal error: Register 'r4': RTOS plugin reports 0 bytes register size, while GDB Server expects 4 bytes
Read register 'r4' (4 bytes) from hardware: 0x65000000

[acassis]

@kistlin did you try to ping Segger too? Since the issue is only happening on Windows, maybe they can give us some hint what is causing the issue. What do you think?