Unicorn Engine Breaks GDB Memory Access Inside Callback

[zachriggle]

It is not possible to read inferior memory within GDB, inside a Unicorn callback.

Does Unicorn do anything that would explain breakage of GDB's functionality when inside of a callback?

I have a minimized PoC here. Just save it as script.py.

$ gdb --nx --batch  -ex "set stop-on-solib-events 1" -ex "run" -ex "source script.py" --args /bin/sh

"""Save this script as script.py"""
import unicorn as U
import gdb

proc = gdb.selected_inferior()

instructions = b'\x90' * 10

uc = U.Uc(U.UC_ARCH_X86, U.UC_MODE_32)

ADDRESS   = 0xdead0000

uc.mem_map(ADDRESS, 2 * 1024 * 1024)
uc.mem_write(ADDRESS, instructions)

def print_memory(msg):
    pc = gdb.parse_and_eval('$pc')
    gdb.execute('x/i $pc')
    print(msg, repr(proc.read_memory(pc, 4).tobytes()))

def hook(uc, address, size, user_data):
    print(uc, hex(address), size, user_data)
    try:
        print_memory("inside")
    except gdb.MemoryError:
        uc.emu_stop()
        raise

uc.hook_add(U.UC_HOOK_CODE, hook)

print_memory("before")
uc.emu_start(ADDRESS, ADDRESS+4)
print_memory("after")

You should see the following output, or something like it. It shows that the read_memory call fails while inside Unicorn. This was done with the most recent master on Unicorn (86823f5).

Stopped due to shared library event (no libraries added or removed)
=> 0x7ffff7ddc08c <dl_main+5756>:       nop
before b'\x90\x8b\x15M'
<unicorn.unicorn.Uc object at 0x7f7fa3109128> 0xdead0000 1 None
=> 0x7ffff7ddc08c <dl_main+5756>:       nop
Traceback (most recent call last):
  File "_ctypes/callbacks.c", line 234, in 'calling callback function'
  File "/usr/local/lib/python3.4/dist-packages/unicorn/unicorn.py", line 329, in _hookcode_cb
    cb(self, address, size, data)
  File "script.py", line 30, in hook
    print_memory("inside")
  File "script.py", line 25, in print_memory
    print(msg, repr(proc.read_memory(pc, 4).tobytes()))
gdb.MemoryError: Cannot access memory at address 0x7ffff7ddc08c
=> 0x7ffff7ddc08c <dl_main+5756>:       nop
after b'\x90\x8b\x15M'

[lunixbochs]

Unicorn callbacks execute on a different OS thread IIRC, which could be related if GDB cares about that.

[zachriggle]

This appears to be the reason. Of course, now the issue is whether it's possible to have Unicorn issue callbacks on the same thread, and to find out why GDB is plenty happy to do almost everything except fetch memory.

~ ❯❯❯ cat foo.py
import threading
import gdb

proc = gdb.selected_inferior()

def print_memory(msg):
    pc = gdb.parse_and_eval('$pc')
    gdb.execute('x/i $pc')
    print(msg, repr(proc.read_memory(pc, 4).tobytes()))

def thread(*a,**kw):
    print_memory("in thread")

print_memory("before")
t=threading.Thread(target=thread)
t.start()
t.join()
print_memory("after")
~ ❯❯❯ gdb --nx --batch  -ex "set stop-on-solib-events 1" -ex "run" -ex "source foo.py" --args /bin/sh
Stopped due to shared library event (no libraries added or removed)
=> 0x7ffff7ddc08c <dl_main+5756>:       nop
before b'\x90\x8b\x15M'
=> 0x7ffff7ddc08c <dl_main+5756>:       nop
Exception in thread Thread-1:
Traceback (most recent call last):
  File "/usr/lib/python3.4/threading.py", line 920, in _bootstrap_inner
    self.run()
  File "/usr/lib/python3.4/threading.py", line 868, in run
    self._target(*self._args, **self._kwargs)
  File "foo.py", line 12, in thread
    print_memory("in thread")
  File "foo.py", line 9, in print_memory
    print(msg, repr(proc.read_memory(pc, 4).tobytes()))
gdb.MemoryError: Cannot access memory at address 0x7ffff7ddc08c

=> 0x7ffff7ddc08c <dl_main+5756>:       nop
after b'\x90\x8b\x15M'

[lunixbochs]

Unicorn runs the CPU in a separate thread. Callbacks are issued synchronously from the CPU thread. You'd need to rewrite parts of Unicorn to move the CPU emulation to the main thread, which is technically possible but probably @aquynh's decision.

Does the GDB memory read need to run synchronously with the callback? I could help you work around this in Python.

[zachriggle]

Yes, it does unfortunately need to be synchronous.

The intent is to emulate the current basic block, such that the next conditional jump (or series of conditional jumps) can be evaluated ahead of time, to provide better disassembly.

This requires using Capstone to inspect the instruction to determine its type (jump vs. call vs. ret).

While I can use uc.read_memory to read from the mapped state of the child process, the mappings are also populated in the same manner in the first place.

    def map_page(self, page):
        page = pwndbg.memory.page_align(page)

        try:
            data = pwndbg.memory.read(page, pwndbg.memory.PAGE_SIZE)
        except gdb.MemoryError:
            print("Could not map page %#x during emulation!" % page)
            return False

        if not data:
            return False

        self.uc.mem_map(page, pwndbg.memory.PAGE_SIZE)
        self.uc.mem_write(page, bytes(data))

        return True

    def hook_mem_invalid(self, uc, access, address, size, value, user_data):

        # Page-align the start address
        start = pwndbg.memory.page_align(address)
        size  = pwndbg.memory.page_size_align(address + size - start)
        stop  = start + size

        # Map each page with the permissions that we think it has.
        for page in range(start, stop, pwndbg.memory.PAGE_SIZE):
            if not self.map_page(page):
                return False

        return True

It appears that the page-mapping requests occur on the first thread, as I don't run into the same problems when using GDB's memory read operation.

Using the emulator's memory state will also introduce lots of architectural problems, versus using GDB's memory state.

[aquynh]

this raises a very good question. yes, Unicorn does not need threading, which we inherited from Qemu, which has async threading for I/O, interrupts, etc.

will be great if somebody can work on this to remove thread requirement in Unicorn.

[lunixbochs]

Removing the extra thread should also improve rapid stop/start performance.

[JonathonReinhart]

The fact that vCPUs run on their own threads is also troublesome for unit testing. cmocka wants to longjmp out of the test when an assertion fails, but that clearly won't work if the setjmp happened on a different thread!

[aquynh]

i put this into the TODO list: https://github.com/unicorn-engine/unicorn/wiki/TODO

[lunixbochs]

Will a single uc handle always only contain one CPU? Is CPU_FOREACH redundant now? If so, I have a patch planned out / in progress.

[lunixbochs]

@zachriggle my PR #481 (https://github.com/lunixbochs/unicorn/commit/f0af8f828226285ce095bc9a736fffba50e54cb5) fixes ptrace in callbacks by moving CPU execution to the same thread as uc_emu_start(). Haven't found any regressions yet, but more testing is encouraged before this is merged. I also need someone to test on Windows.

[lunixbochs]

@zachriggle this is fixed in master by the no-thread merge right?

[aquynh]

Solved, thanks