Linux LLDB fails to update the value of bytes which have breakpoints on it when we write to the memory

[xusheng6]

On Linux, when writing to memory, the bytes where there are breakpoints on it are left unchanged. The expected behavior is to report the newly written value -- despite the actual byte value is 0xc3.

$ lldb '/mnt/binja/clion/binaryninja/public/debugger/test/binaries/Linux-x86_64/helloworld' 
(lldb) target create "/mnt/binja/clion/binaryninja/public/debugger/test/binaries/Linux-x86_64/helloworld"
Current executable set to '/mnt/binja/clion/binaryninja/public/debugger/test/binaries/Linux-x86_64/helloworld' (x86_64).
(lldb) b main
Breakpoint 1: where = helloworld`main, address = 0x0000000000001169
(lldb) r
Process 134979 launched: '/mnt/binja/clion/binaryninja/public/debugger/test/binaries/Linux-x86_64/helloworld' (x86_64)
Process 134979 stopped
* thread #1, name = 'helloworld', stop reason = breakpoint 1.1
    frame #0: 0x0000555555555169 helloworld`main
helloworld`main:
->  0x555555555169 <+0>: endbr64 
    0x55555555516d <+4>: pushq  %rbp
    0x55555555516e <+5>: movq   %rsp, %rbp
    0x555555555171 <+8>: subq   $0x20, %rsp
(lldb) memory write 0x555555555169 -s 4 aaaaaaaa
(lldb) x/4xw 0x555555555169
0x555555555169: 0xaaaaaaf3 0xe5894855 0x20ec8348 0x48ec7d89

It can be seen the first four bytes become 0xaaaaaaf3 after the memory write, instead of the expected value 0xaaaaaaaa.

[llvmbot]

@llvm/issue-subscribers-lldb

[dwblaikie]

I noticed you filed this separately for Linux and Windows - do you know if this works on any lldb platform (say, on MacOS) - or is it just generally broken for lldb in general/might have a general lldb fix?

[xusheng6]

@dwblaikie I think it is a platform issue. It actually behaves differently on Linux and Windows. On Windows, the byte is not shadowed at all, i.e., if I need the byte that has a breakpoint on it, I get 0xc3. On Linux, the byte is shadowed, and a memory read never returns 0xc3. However, suppose the byte is initially 0xf3, after I write 0xaa to it, the memory read will still return 0xf3, instead of the expected 0xaa.

Things work correctly on macOS.

[jimingham]

It's not surprising this is platform dependent. On Linux, lldb delegates setting breakpoints to lldb-server, and on Darwin debugserver does that job. In both cases, lldb doesn't directly set traps, it just sends the "set me a breakpoint at this address" gdb-remote protocol packet. So lldb never knows how the breakpoint is implemented. Because of that, lldb-server or debugserver have to be responsible for showing lldb the original memory contents with the traps removed. lldb is on purpose unaware of these details.

There is also support for inserting and removing the traps directly in lldb in case you are talking to a stub that doesn't support the breakpoint setting packets. But most stubs these days support breakpoint setting with the Z packet, so it's probably less well exercised. Windows used to be the only platform that didn't use a server-stub, but instead debugged directly from lldb. IIRC there were discussion about switching to lldb-server on Windows, but I don't know if that's happened yet.

Anyway, if you are seeing traps in "memory read" or the output of Process::ReadMemory then that's a bug in the the direct trap handling support. If you are just seeing the traps in the lower level direct memory reads, but they get replaced by the backing memory in the higher level commands and Process::ReadMemory, then that's as expected.

On a side note, the debugger is always going to have problems supporting programs that write over its breakpoint locations. You can't effectively watch all the memory regions where you've set the breakpoint so if the program writes over our trap instruction, that breakpoint will cease to function. There's really nothing we can do about that. You can patch this up the next time you stop for some other reason by reading the memory for all the breakpoints on each stop to see if they might have changed, and if so reinserting the trap and refreshing the backing store. We don't do that currently since on most modern systems you can only execute out of read-only memory, so checking for whether somebody scribbled over one of our breakpoints is a fair bit of traffic between lldb & the stub for very little benefit.

Jim

On Jan 18, 2023, at 5:24 AM, xusheng @.***> wrote:

@dwblaikie https://github.com/dwblaikie I think it is a platform issue. It actually behaves differently on Linux and Windows. On Windows, the byte is not shadowed at all, i.e., if I need the byte that has a breakpoint on it, I get 0xc3. On Linux, the byte is shadowed, and a memory read never returns 0xc3. However, suppose the byte is initially 0xf3, after I write 0xaa to it, the memory read will still return 0xf3, instead of the expected 0xaa.

— Reply to this email directly, view it on GitHub https://github.com/llvm/llvm-project/issues/60062#issuecomment-1387069760, or unsubscribe https://github.com/notifications/unsubscribe-auth/ADUPVW7UQLDKEGAIN6HFXQLWS7VKLANCNFSM6AAAAAAT4OB4HQ. You are receiving this because you are on a team that was mentioned.