search

epasveer / seer

Seer - a gui frontend to gdb
GNU General Public License v3.0
2.13k stars 69 forks source link

Program is unusable when trying to debug kernel #119

Closed RadxaYuntian closed 10 months ago

RadxaYuntian commented 1 year ago

I'm using agent-proxy with nokaslr kgdbwait console=ttyS2,460800n8 kgdboc=ttyS2,460800n8 kernel cmdline to debug an embedded platform. I can successfully attach Seer with seergdb --connect localhost:6443 vmlinux. However, from agent-proxy's console port, I can see many gdb data being transmitted, and the execution control commands in Seer becomes unusable.

epasveer commented 1 year ago

Hi Zhang. (Thanks for trying out Seer).

I admit I need to improve embedded debugging.

What other debuggers have you had success with? I can look to see what they do.

Thanks.

RadxaYuntian commented 1 year ago

Unfortunately none so far. I tried gdbfrontend and it can't even properly attached to it. gdbgui's AUR package is currently broken so I didn't bother. Plain gdb can attach and execute continue command. However, my breakpoint was not hit for whatever reason, which is why I wanted to verify with a GUI front end to rule out user error. Right now I'm back to printf...

epasveer commented 1 year ago

Seer relies on gdb. If gdb can't do it natively, Seer won't be any additional help here.

RadxaYuntian commented 1 year ago

I understand that. I just want to use seer to verify that the issue is indeed gdb and not user error. However, right now I cannot let kernel continue execution in seer, which was not an issue when using gdb directly.

epasveer commented 10 months ago

Hi,

Just checking if you've had better success with native gdb with your debugging? Or perhaps another frontend has come along to help you?

There have been improvements to Seer's handling of connecting to a 'gdbserver' type process. You may want to give Seer another attempt by grabbing the latest "main" from github.

$ seergdb --connect localhost:6443 --sym vmlinux

It may have the continue function fixed.

RadxaYuntian commented 10 months ago

I just happen to have another driver related tasks today, and I'll see if I could find an excuse to use debugger :smile:

epasveer commented 10 months ago

To help out. First run seer by itself and enable the gdb output and seer output to the log windows.

image image

Then run Seer again when you attempt to debug. The two log views have "save-to-file" buttons. When it disconnects, save the two log files and attach it to this task. It would be greatly appreciated. It will help me debug the problem (hopefully).

RadxaYuntian commented 10 months ago

Here is how I do it with note to future me. Targeting device is Radxa ROCK 4A which is based on Rockchip RK3399.

  1. Configure kernel cmdline with kgdbwait nokaslr kgdboc=ttyS2,460800n8 console=ttyS2,460800n8 to enable debugging. ttyFIQ0 is not supported, and 1.5M baud does not work.
  2. Boot and observe that the serial output stops with KGDB: Waiting for connection from remote gdb....
  3. Exit terminal emulator, since we will use agent-proxy to split console data with debugger data, and that requires exclusive access to the serial device.
  4. Run agent-proxy 5550^5551 0 /dev/ttyUSB0,460800
  5. telnet localhost 5550 to interact with serial console.
  6. In Linux source code folder, aarch64-linux-gnu-gdb arch/arm64/boot/Image to enter gdb, then run tar rem localhost:5551 to attach to remote target.
  7. Run c command to continue.

Seer specific part:

  1. Settings->Configuration->GDB and set to /usr/bin/aarch64-linux-gnu-gdb (platform dependent).
  2. Reboot the board.
  3. seergdb --connect localhost:5551
  4. Seer slowly communicates with kgdb...
  5. Once it is finish, I clicked Continue and it actually continued execution! So the issue is resolved.

I'll still attach the log in case you can see how to improve the wait time between connection established and control flow available.

Archive.tar.gz

RadxaYuntian commented 10 months ago

Although it does not seem like I can interrupt the execution flow again with Ctrl+I. I'm not familiar with gdb or Linux kernel debugging in general, so that could be some misconfiguration on my end.

epasveer commented 10 months ago

Glad to hear it works. And thanks for the archive file.

The initial communication that Seer is doing can be related to two things:

  1. Loading the names of the source files from the executable image. How many source files show up in the Source view?
  2. The number of threads. Your debug session shows 84 threads. Is that correct?

Btw, thanks for the details of what steps you do to get things going. It helps me to see the "bigger picture" of this type of debugging.

RadxaYuntian commented 10 months ago

I do not see any source files in the source view and that's likely because I didn't have a symbol file generated with my kernel. I also can't answer if the thread count is correct since I haven't been able to get the debugging environment set up.

epasveer commented 10 months ago

Okay. Thanks for the feedback.

in case you can see how to improve the wait time between connection established and control flow available.

I'll look at the logs for that. gdb has some internal settings to enable timing of operations. I haven't played with it yet. I'll create a separate task for myself to look into it.

Thanks again for trying Seer.