Open nicholasbishop opened 3 years ago
I found this article, but their approach doesn't seem to work. It finds the base address and address of each section, but still doesn't provide any symbols to gdb. Then I found a uefi_test_runner_xxxx.pdb
file alongside a uefi_test_runner_xxxx.efi
file in target/{arch}/debug/deps/
, but lldb doesn't seem to want to load it, maybe the location of the .pdb
isn't in the executable? I'm starting to lean towards it being a toolchain issue. I'll try building EDK2 and OVMF using their LLVM/Clang toolchain and see if it outputs usable debug info first.
Previous versions of rustc
and/or cargo
worked flawlessly as described in the article above, but this doesn't work with recent version. I don't remember whether the .pdb
files were always there, but in any case they don't include any symbols now.
IIRC, the spec requires that UEFI apps tick a few boxes, e.g. the subsystem type, but it also states that they ought to be stripped, hence this behavior.
The executable should be stripped, but applications built in C with EDK2 in DEBUG
mode do output .map
and .pdb
files for debugging. Source: https://edk2-docs.gitbook.io/edk-ii-build-specification/12_build_changes_and_customizations/121_building_for_debug
With that in mind, it sounds like this might be worth filing a bug for the x86_64-unknown-uefi
target?
I'm in the process of bisecting compiler versions to determine where the old behavior of having symbol tables was lost.
The last build that provides symbol tables is nightly-2020-11-14-x86_64-unknown-linux-gnu
, while nightly-2020-11-15-x86_64-unknown-linux-gnu
doesn't.
I'll try to do some digging as to why that happens, but I don't know the first thing about rustc or cargo code so I don't expect to find the solution TBH.
I took uefi-test-runner
@ v0.6.0
as a testbed. Rust nightlies tried ranged from 2020-10-20
to 2021-10-04
. For getting versions of Rust, I used these commands, substituting the dates accordingly:
rustup toolchain add nightly-2020-11-14 --profile minimal
rustup default nightly-2020-11-14
rustup component add rust-src
For building, ./build.py build
was used. In order to detect whether symbol tables are present, the following commands were run in gdb via a .gdbinit
file:
add-symbol-file ../target/x86_64-unknown-uefi/debug/uefi-test-runner.efi
info functions
Bisecting leads to rust-lang/rust#78959, which is also linked to by rust-lang/rust#87157 where the author of the blog post above reports the same issue.
The solution seems to be to do rustc +nightly -Z unstable-options --print target-spec-json --target x86_64-unknown-uefi | sed -e 's/"is-like-msvc": true/"is-like-msvc": false/g' > x86_64-unknown-uefi-debug.json
.
Can confirm the above solution works, with the additional step of removing "is-builtin" from the target file. build.py
would need to be updated to use the custom target as well.
I'll dive into debugging something in QEMU tonight, but this looks like it solves the issue completely as gdb can see all symbols and map them back to the source files.
I'm not seeing quite the same thing as you with the builtin x86_64-unknown-uefi target. I do get what seems to be a valid pdb file generated under the deps
directory. I dumped the symbols in it with the pdb_symbols
utility in the pdb crate, and it looks correct. (rustc 1.57.0-nightly (9dbb26efe 2021-10-03
.)
So if I'm understanding correctly, rustc used to include DWARF-style debug sections in the executable, and now it outputs a pdb. Which does seem like reasonable behavior.
lldb seems like it supports loading pdb files (either with a Windows-specific loader, or the "native pdb" plugin on other platforms), but in testing on Linux I haven't had any luck getting it to successfully load symbols.
Yeah, that's the same issue that I was having with the pdb files. I tested on both linux and windows(mingw) and was unable to correctly load any symbols. I'll give MSVC a try.
With @no92's solution, gdb and lldb in linux are able to recognize the symbols enough to say they have successfully set a breakpoint, but they won't actually break with br
or hbr
. Maybe the addresses aren't correct and need to be fixed up before debugging?
EDIT: Tried to fixup the addresses using the above article, but it segfaulted while running on my machine.
@nicholasbishop wrote:
I do get what seems to be a valid pdb file generated under the deps directory. I dumped the symbols in it with the pdb_symbols utility in the pdb crate, and it looks correct.
I had checked it with some pdb tool I frankly can't remember right now, and I didn't find anything resembling symbols. It was, however, a valid pdb file. As it was some random tool pulled from GitHub, it could just be wrong and the Rust crate right. Oops.
@timrobertsdev wrote:
With @no92's solution, gdb and lldb in linux are able to recognize the symbols enough to say they have successfully set a breakpoint, but they won't actually break with br or hbr. Maybe the addresses aren't correct and need to be fixed up before debugging?
As EFI apps are relocatable, the base address changes. This could be read out via the LoadedImage Protocol, but transferring that info from the remote to the debugger host would have to take place somehow. Alternatively, the instructions in the article work for me, I'm not sure why it's segfaulting for you. I use the python script via a .gdbinit
in gdb itself as well as some VSCode interface that ultimately uses gdb, too.
@no92 Nevermind, I got it working. The mistake I was making was attempting to run the dbg
gdbscript while QEMU itself was halted and waiting for debugger, instead of the wait_for_debugger()
fn outlined in that article.
I believe I've taken all the steps mentioned above yet the problem persists -- GDB is not able to recognize any symbols (which I doubt exist). I might've missed something. Any help would be much appreciated.
What I did:
nightly-2020-11-14
(used rustup default and updated rust-toolchain.toml
).cargo/config
file and was indeed recognized)While I haven't used this in a while, rolling back to an old nightly version should not be necessary. Using the script provided in the blog post linked above is necessary IIRC.
Unfortunately, it still doesn't work for me. We're talking about load_symbols.py
, right?
I have successfully solved the problem by adding the following build (under the [build] section) flag in .cargo/config:
"-C", "link-args=/debug:dwarf"
Thanks to the following video: https://www.youtube.com/watch?v=2YAgDJTs9So
This might be trivial to some however I wasn't aware of this link argument in the first place. All symbols are now recognized by gdb.
How did you get it working with gdb? I can't seem to figure it out
"-C", "link-args=/debug:dwarf"
This enables for example objdump
to displays debug symbols. gdb
, however, still doesn't find any symbols in the corresponding efi
file.
I've been digging around this issue recently, here's what I've discovered so far. I've been trying to use LLDB, since it is the only debugger which can read PDB files (in theory) and connect to QEMU using GDB's remote debug protocol.
I've tried to follow along with the instructions from this article on the osdev.org wiki. I've modified the xtask/src/qemu.rs
file to pass the -s
flag to QEMU (i.e. I've added a line with cmd.arg("-s");
) in order to activate the GDB debug port, then I've modified the uefi-test-runner/src/main.rs
file to display the address at which the binary is loaded:
{
let loaded_image = st.boot_services()
.open_protocol_exclusive::<LoadedImage>(image)
.expect("Failed to open LoadedImage protocol");
println!("Image base: {:#X}", loaded_image.info().0 as usize);
}
let wait = true;
while wait { }
When running the app with cargo xtask run
, it displays a message similar to Image base: 0xDD89000
.
Now, I ran the following commands in LLDB, from the target/x86_64-unknown-uefi/debug/deps
directory:
target create uefi_test_runner-e4bed4b0ac1a4cab.efi
— loads the EFI binary and sets it as the default debug target.target symbols add uefi_test_runner-e4bed4b0ac1a4cab.pdb
— loads the debug symbols from the PDB file.gdb-remote localhost:1234
— connects to the running QEMU instance (Note: sometimes I had to run it twice to get it to work).process continue
— allow the code to run until it gets into the infinite loop we've set up in the main function.process interrupt
— pause inside the while
loop inside the main function.image load --file uefi_test_runner-e4bed4b0ac1a4cab.efi .text 0xDD89000
— tell LLDB at which address the EFI binary's .text
section starts.At this point, everything should be set up for debugging. Unfortunately, LLDB just crashes:
PLEASE submit a bug report to https://github.com/llvm/llvm-project/issues/ and include the crash backtrace.
Stack dump:
0. Program arguments: lldb
#0 0x00007f9ba8e3fd01 llvm::sys::PrintStackTrace(llvm::raw_ostream&, int) (/lib/x86_64-linux-gnu/libLLVM-14.so.1+0xe3fd01)
#1 0x00007f9ba8e3da3e llvm::sys::RunSignalHandlers() (/lib/x86_64-linux-gnu/libLLVM-14.so.1+0xe3da3e)
#2 0x00007f9ba8e40236 (/lib/x86_64-linux-gnu/libLLVM-14.so.1+0xe40236)
#3 0x00007f9ba7842520 (/lib/x86_64-linux-gnu/libc.so.6+0x42520)
#4 0x00007f9bb2ca5f75 (/lib/x86_64-linux-gnu/liblldb-14.so.1+0xaa5f75)
#5 0x00007f9bb2caa78e (/lib/x86_64-linux-gnu/liblldb-14.so.1+0xaaa78e)
#6 0x00007f9bb2cab262 (/lib/x86_64-linux-gnu/liblldb-14.so.1+0xaab262)
#7 0x00007f9bb2ca6c91 (/lib/x86_64-linux-gnu/liblldb-14.so.1+0xaa6c91)
#8 0x00007f9bb2ca6d32 (/lib/x86_64-linux-gnu/liblldb-14.so.1+0xaa6d32)
#9 0x00007f9bb2caa613 (/lib/x86_64-linux-gnu/liblldb-14.so.1+0xaaa613)
#10 0x00007f9bb2cab2ef (/lib/x86_64-linux-gnu/liblldb-14.so.1+0xaab2ef)
#11 0x00007f9bb2ca6c91 (/lib/x86_64-linux-gnu/liblldb-14.so.1+0xaa6c91)
#12 0x00007f9bb2cab7a0 (/lib/x86_64-linux-gnu/liblldb-14.so.1+0xaab7a0)
#13 0x00007f9bb2cab3d6 (/lib/x86_64-linux-gnu/liblldb-14.so.1+0xaab3d6)
#14 0x00007f9bb2ca6c91 (/lib/x86_64-linux-gnu/liblldb-14.so.1+0xaa6c91)
#15 0x00007f9bb2c88d16 (/lib/x86_64-linux-gnu/liblldb-14.so.1+0xa88d16)
#16 0x00007f9bb2c8936d (/lib/x86_64-linux-gnu/liblldb-14.so.1+0xa8936d)
#17 0x00007f9bb2c8587b (/lib/x86_64-linux-gnu/liblldb-14.so.1+0xa8587b)
#18 0x00007f9bb2c85263 (/lib/x86_64-linux-gnu/liblldb-14.so.1+0xa85263)
#19 0x00007f9bb2c8b078 (/lib/x86_64-linux-gnu/liblldb-14.so.1+0xa8b078)
#20 0x00007f9bb260e6ec (/lib/x86_64-linux-gnu/liblldb-14.so.1+0x40e6ec)
#21 0x00007f9bb25de1ba (/lib/x86_64-linux-gnu/liblldb-14.so.1+0x3de1ba)
#22 0x00007f9bb25de004 (/lib/x86_64-linux-gnu/liblldb-14.so.1+0x3de004)
#23 0x00007f9bb27f5f1e (/lib/x86_64-linux-gnu/liblldb-14.so.1+0x5f5f1e)
#24 0x00007f9bb27f5b90 (/lib/x86_64-linux-gnu/liblldb-14.so.1+0x5f5b90)
#25 0x00007f9bb27e8d82 (/lib/x86_64-linux-gnu/liblldb-14.so.1+0x5e8d82)
#26 0x00007f9bb27ea36f (/lib/x86_64-linux-gnu/liblldb-14.so.1+0x5ea36f)
#27 0x00007f9bb2781b2f (/lib/x86_64-linux-gnu/liblldb-14.so.1+0x581b2f)
#28 0x00007f9bb2783c29 (/lib/x86_64-linux-gnu/liblldb-14.so.1+0x583c29)
#29 0x00007f9bb27ba671 (/lib/x86_64-linux-gnu/liblldb-14.so.1+0x5ba671)
#30 0x00007f9bb274184c (/lib/x86_64-linux-gnu/liblldb-14.so.1+0x54184c)
#31 0x00007f9bb25eb03f (/lib/x86_64-linux-gnu/liblldb-14.so.1+0x3eb03f)
#32 0x00007f9bb26c1912 (/lib/x86_64-linux-gnu/liblldb-14.so.1+0x4c1912)
#33 0x00007f9bb260a83f (/lib/x86_64-linux-gnu/liblldb-14.so.1+0x40a83f)
#34 0x00007f9bb25eb73c (/lib/x86_64-linux-gnu/liblldb-14.so.1+0x3eb73c)
#35 0x00007f9bb26c33d9 (/lib/x86_64-linux-gnu/liblldb-14.so.1+0x4c33d9)
#36 0x00007f9bb23f5eda lldb::SBDebugger::RunCommandInterpreter(bool, bool) (/lib/x86_64-linux-gnu/liblldb-14.so.1+0x1f5eda)
#37 0x0000000000407d4a (/usr/lib/llvm-14/bin/lldb+0x407d4a)
#38 0x0000000000408f85 (/usr/lib/llvm-14/bin/lldb+0x408f85)
#39 0x00007f9ba7829d90 __libc_start_call_main ./csu/../sysdeps/nptl/libc_start_call_main.h:58:16
#40 0x00007f9ba7829e40 call_init ./csu/../csu/libc-start.c:128:20
#41 0x00007f9ba7829e40 __libc_start_main ./csu/../csu/libc-start.c:379:5
#42 0x0000000000404125 (/usr/lib/llvm-14/bin/lldb+0x404125)
Segmentation fault (core dumped)
I'm guessing it tries to parse the PDB file and the current stack frame, but it triggers some bug somewhere.
I haven't submitted a bug report to LLDB yet, I'm going to play around with this setup for a bit longer.
@GabrielMajeri I managed to replicate your process in VSCode and LLDB shell as well
The pdb files not being copied is a cargo bug: https://github.com/rust-lang/cargo/pull/5179 did not include -uefi as a target to copy pdbs for. I don't feel like filing an actual bug right now; I'm presently testing just patching it and I may file a PR later.
@GabrielMajeri This may be because the virtual address of .text in your image is not 0. You can check the virtual addresses of all sections by using llvm-readobj --sections <efi>
. For instance, assumes it prints Image base: 0xDD89000
and
llvm-readobj --sections ./lab/esp/efi/boot/bootx64.efi
File: ./lab/esp/efi/boot/bootx64.efi
Format: COFF-x86-64
Arch: x86_64
AddressSize: 64bit
Sections [
Section {
Number: 1
Name: .text (2E 74 65 78 74 00 00 00)
VirtualSize: 0x11B66
VirtualAddress: 0x1000 <-------------------------------
RawDataSize: 72704
PointerToRawData: 0x400
PointerToRelocations: 0x0
PointerToLineNumbers: 0x0
RelocationCount: 0
LineNumberCount: 0
Characteristics [ (0x60000020)
IMAGE_SCN_CNT_CODE (0x20)
IMAGE_SCN_MEM_EXECUTE (0x20000000)
IMAGE_SCN_MEM_READ (0x40000000)
]
}
......
Then .text
shoud be set to 0xDD8A000
.
After a week of struggle, I finally realized that using LLDB to debug UEFI applications is far from mature. And I still cannot let the rustc and lld to output dwarf debug symbols. The line below seems to have no effect on me.
"-C", "link-args=/debug:dwarf"
The following addition in .cargo/config.toml
just causes the .pdb file to no longer be generated and from what I can see does not add any debug sections to the .efi file. It seems to give you function names but for example variable names remained unnamed.
[target.x86_64-unknown-uefi]
rustflags = ["-C", "link-args=/debug:dwarf"]
The only solution seems to be the following as mention earlier. Hopefully it will help someone in the future.
The solution seems to be to do
rustc +nightly -Z unstable-options --print target-spec-json --target x86_64-unknown-uefi | sed -e 's/"is-like-msvc": true/"is-like-msvc": false/g' > x86_64-unknown-uefi-debug.json
.
I also needed the following additions to my .cargo/config.toml
[unstable]
build-std = ["core", "compiler_builtins", "alloc"]
build-std-features = ["compiler-builtins-mem"]
With that the target (x86_64-unknown-uefi-debug.json) can then be built with nightly
cargo +nightly build --target x86_64-unknown-uefi-debug.json
The resulting .efi file will then have debug info which will work with gdb
Sections:
Idx Name Size VMA LMA File off Algn
0 .text 0001a098 0000000140001000 0000000140001000 00000400 2**4
CONTENTS, ALLOC, LOAD, READONLY, CODE
1 .rdata 00005e5c 000000014001c000 000000014001c000 0001a600 2**4
CONTENTS, ALLOC, LOAD, READONLY, DATA
2 .data 00000050 0000000140022000 0000000140022000 00020600 2**4
CONTENTS, ALLOC, LOAD, DATA
3 .eh_fram 00000040 0000000140023000 0000000140023000 00020800 2**2
CONTENTS, ALLOC, LOAD, READONLY, DATA
4 .reloc 000003bc 0000000140024000 0000000140024000 00020a00 2**2
CONTENTS, ALLOC, LOAD, READONLY, DATA
5 .debug_abbrev 00009bc1 0000000140025000 0000000140025000 00020e00 2**0
CONTENTS, READONLY, DEBUGGING
6 .debug_aranges 00020bc0 000000014002f000 000000014002f000 0002aa00 2**0
CONTENTS, READONLY, DEBUGGING
7 .debug_info 00172453 0000000140050000 0000000140050000 0004b600 2**0
CONTENTS, READONLY, DEBUGGING
8 .debug_line 0009145e 00000001401c3000 00000001401c3000 001bdc00 2**0
CONTENTS, READONLY, DEBUGGING
9 .debug_loc 000088d9 0000000140255000 0000000140255000 0024f200 2**0
CONTENTS, READONLY, DEBUGGING
10 .debug_pubnames 0006f820 000000014025e000 000000014025e000 00257c00 2**0
CONTENTS, READONLY, DEBUGGING
11 .debug_pubtypes 0009f74d 00000001402ce000 00000001402ce000 002c7600 2**0
CONTENTS, READONLY, DEBUGGING
12 .debug_ranges 00050ec0 000000014036e000 000000014036e000 00366e00 2**0
CONTENTS, READONLY, DEBUGGING
13 .debug_str 001d3c4c 00000001403bf000 00000001403bf000 003b7e00 2**0
CONTENTS, READONLY, DEBUGGING
Splitting this out from https://github.com/rust-osdev/uefi-rs/issues/285:
@timrobertsdev wrote:
I don't know the answer to this, but it would be nice to have a straightforward way to debug an application built with uefi-rs running under qemu. I'm not sure if there is something that currently works and it just needs some documentation, or if there's work needed in the toolchain.