Closed Quuxplusone closed 3 years ago
Attached full-logs.txt
(5922 bytes, text/plain): Full logs from LLDB 6.0, LLDB 5.0 and LLDB 4.0
Reproducible with current LLDB master, which will be 9.0, e.g. running this
docker container (with extra params to allow debugging):
$ docker run --cap-add=SYS_PTRACE --security-opt seccomp=unconfined --security-
opt apparmor=unconfined weliveindetail/lldb-bug36209:1dbd42ab
+ cd bug36209
+ /llvm-debug/bin/clang --version
clang version 9.0.0 (https://github.com/llvm/llvm-project.git
1dbd42ab5babff6329d4f756b5f2237a2937ddb6)
Target: x86_64-unknown-linux-gnu
Thread model: posix
InstalledDir: /llvm-debug/bin
+ /llvm-debug/bin/clang++ -g -S -emit-llvm -o hellow.ll hellow.cpp
+ /llvm-debug/bin/lli hellow.ll
Hello World!
+ /llvm-debug/bin/lldb -b -s lldb_cmds -- /llvm-debug/bin/lli
(lldb) target create "/llvm-debug/bin/lli"
Current executable set to '/llvm-debug/bin/lli' (x86_64).
(lldb) command source -s 0 'lldb_cmds'
Executing commands in '/bug36209/lldb_cmds'.
(lldb) settings set plugin.jit-loader.gdb.enable on
(lldb) log enable lldb jit
(lldb) b jitbp
Breakpoint 1: no locations (pending).
WARNING: Unable to resolve breakpoint to any actual locations.
(lldb) r hellow.ll
lldb JITLoaderGDB::SetJITBreakpoint looking for JIT register hook
lldb JITLoaderGDB::SetJITBreakpoint looking for JIT register hook
lldb JITLoaderGDB::SetJITBreakpoint looking for JIT register hook
lldb JITLoaderGDB::SetJITBreakpoint setting JIT breakpoint
intern-state JITLoaderGDB::JITDebugBreakpointHit hit JIT breakpoint
intern-state JITLoaderGDB::ReadJITDescriptorImpl registering JIT entry at
0x3b0ffb0 (2712 bytes)
warning: failed to set breakpoint site at 0x4 for breakpoint 1.1: error: 0
sending the breakpoint request
1 location added to breakpoint 1
warning: failed to set breakpoint site at 0x4 for breakpoint 1.1: error: 0
sending the breakpoint request
Hello World!
Process 39 exited with status = 0 (0x00000000)
Image runs Ubuntu 18.04 and LLDB built from https://github.com/llvm/llvm-
project/commit/1dbd42ab.
https://reviews.llvm.org/D61611 did a first step by fixing the "sending the
breakpoint request" error. Now LLDB again manages to implement breakpoint sites
in JITed code.
The next part of the regression is that we only see disassembly, while LLDB 5.0
was successfully loading and showing the respective source lines.
In a last step we should update the documentation:
http://llvm.org/docs/DebuggingJITedCode.html
This is exciting. Thanks Stefan!
I tried to fix source-level debugging in RuntimeDyldMachO a while back but was not able to wrap my head around the LLDB side of the change (MacOS had been explicitly disabled there) in the time I had. If/when you get this running I would love to go over the protocol with you: hopefully I can write a jit-debugging plugin for JITLink.
Some more investigation showed that the DWARF has some corrupt data. All fields
of type DW_FORM_strp have a 0 offset. That prevents functions and compile units
from being registered correctly. Fields with other data types look fine (e.g.
32 bit unsigned for DW_AT_language).
In a debug session for the regression test I created in D61611
(https://github.com/llvm/llvm-
project/blob/87aa9c9e4d41ed88/lldb/test/Shell/Breakpoint/jitbp_elf.test) I put
a breakpoint in DWARFFormValue::AsCString() once the JIT descriptor was
registered and managed to twist the relevant fields to their correct values:
- DWARFAttributes of the first DW_TAG_subprogram DIE:
DW_AT_name=jitbp
DW_AT_linkage_name=_Z5jitbpv
- DWARFUnit DIE:
DW_AT_name=/path/to/llvm-project/lldb/test/Shell/Breakpoint/Inputs/jitbp.cpp
DW_AT_comp_dir=/path/to/llvm-build/tools/lldb/test/Breakpoint
As a result the jitbp function was added as a base-name and Symtab parsed of
the compile unit's DIEs in detail in order to figure out the source code
location.
It looked promising, until ResolveAddressUsingFileSections() failed to map the
jitbp's AddressRange to one of the compile unit's sections. The range is
DW_AT_low/high_pc == 0/3, which looked reasonable for a section offset. I did
not go into more detail here as this is all standard LLDB logic. I would be a
little surprised if there is a bug here that kills us. I had expected something
like an incorrect type setting similar to D61611.
So, I wonder whether it's really the debugger side that's broken here or if it
might be MCJIT. Not sure it's worth the time to investigate that. Maybe I can
check again with ORCv2 once we have support for listeners.
These fields are empty because the object that we get from MCJIT doesn't have relocations applied. There's multiple reasons for it on the MCJIT side:
lli doesn't set the ProcessAllSections
flag for MCJIT, so
relocations in sections like debug_info are ignored currently.
notifyObjectLoaded() is called before we resolve relocations, thus we always copy over the unrelocated in-memory object.
I can think of two ways that this broke since 4.0:
(1) Either LLDB used to resolve relocations for objects it received via the GDB JIT Interface itself and it stopped doing so between releases 5 and 6.
(2) Or otherwise this has always been lli's responsibility and it broke silently because the feature wasn't used widely anyway.
I think version (1) is more likely, so I will go find a lldb-5.0 and test debugging with top-of-tree lli.
https://reviews.llvm.org/D90769 aims to fix the relocation issue and bring back source-level debugging.
Hi Stefan,
FWIW OrcV2 now supports listeners when using RTDyldObjectLinkingLayer if you want to test this under OrcV2. (I believe there is an example in llvm/examples/OrcV2Examples).
I'm on the road the next few days, but will try to check that review soon. Please ping me if you don't hear anything by next week.
-- Lang.
Yes, I want to do that. For the LLDB test case, however, it doesn't matter so much as it currently uses the same RuntimeDyld functionality under the hood as MCJIT does.
Also (note to myself) I have to patch lli first, as it currently makes the distinction whether or not to add the GDB listener based on the triple's isOSBinFormatELF(). That disqualifies JITing ELF on macOS or Windows even though it's quite a popular workaround.
Part 2 of the fix has landed. It will ship in the upcoming release 12 of LLDB. I also updated the documentation and set up a review: https://reviews.llvm.org/D90789
Just confirmed this to work with RuntimeDyld-based LLJIT on current tip-of-tree:
(lldb) run tools/lldb/test/Breakpoint/Output/jitbp_elf.test.tmp.ll
Process 89272 launched: '/workspaces/llvm-
lldb/build/bin/LLJITWithGDBRegistrationListener' (x86_64)
1 location added to breakpoint 1
Process 89272 stopped
* thread #1, name = 'LLJITWithGDBReg', stop reason = breakpoint 1.1
frame #0: 0x00007ffff7fd0000 JIT(0x43fa720)`jitbp() at jitbp.cpp:1:15
-> 1 int jitbp() { return 0; }
2 int main() { return jitbp(); }
I guess we cannot add a test for it based on the OrcV2 example executable, so
we'd have to add something like an "orc-eager" kind to lli.
@Lang: Do you think that's worth it? Anything special I should consider in case
I decided to do that?
The documentation update landed and should be in the LLVM 12 release. A orc-greedy mode was considered a good addition for LLI in https://reviews.llvm.org/D90789#2491473
full-logs.txt
(5922 bytes, text/plain)