search

qilingframework / qiling

A True Instrumentable Binary Emulation Framework
https://qiling.io
GNU General Public License v2.0
5.14k stars 744 forks source link

Qiling fails to run /bin/ls #1349

Open NudistBeaaach opened 1 year ago

NudistBeaaach commented 1 year ago

Running Qiling on a simple binary is crashing, it throws something about the libc version and in this case a problem about an Operation not permitted which is really weird

from qiling import Qiling
from qiling.const import QL_VERBOSE

def disas():
    ql = Qiling([r'/usr/bin/ls'], r'/home/kali/Desktop/404_RE/qiling/examples/rootfs/x8664_linux')
    ql.run()

if __name__ == "__main__":
    disas()

| v

[=]     brk(inp = 0x0) = 0x55555557c000
[=]     uname(buf = 0x80000000d9c0) = 0x0
[=]     access(path = 0x7ffff7df6082, mode = 0x0) = -0x1 (EPERM)
[=]     access(path = 0x7ffff7df8dd0, mode = 0x4) = -0x1 (EPERM)
[=]     openat(fd = 0xffffff9c, path = 0x7ffff7df6428, flags = 0x80000, mode = 0x0) = -0x1 (EPERM)
[=]     openat(fd = 0xffffff9c, path = 0x80000000d350, flags = 0x80000, mode = 0x0) = -0x1 (EPERM)
[=]     stat(path = 0x80000000d350, buf_ptr = 0x80000000d410) = -0x2 (ENOENT)
[=]     openat(fd = 0xffffff9c, path = 0x80000000d350, flags = 0x80000, mode = 0x0) = -0x1 (EPERM)
[=]     stat(path = 0x80000000d350, buf_ptr = 0x80000000d410) = -0x2 (ENOENT)
[=]     openat(fd = 0xffffff9c, path = 0x80000000d350, flags = 0x80000, mode = 0x0) = -0x1 (EPERM)
[=]     stat(path = 0x80000000d350, buf_ptr = 0x80000000d410) = -0x2 (ENOENT)
[=]     openat(fd = 0xffffff9c, path = 0x80000000d350, flags = 0x80000, mode = 0x0) = -0x1 (EPERM)
[=]     stat(path = 0x80000000d350, buf_ptr = 0x80000000d410) = -0x2 (ENOENT)
[=]     openat(fd = 0xffffff9c, path = 0x80000000d350, flags = 0x80000, mode = 0x0) = -0x1 (EPERM)
[=]     stat(path = 0x80000000d350, buf_ptr = 0x80000000d410) = -0x2 (ENOENT)
[=]     openat(fd = 0xffffff9c, path = 0x80000000d350, flags = 0x80000, mode = 0x0) = -0x1 (EPERM)
[=]     stat(path = 0x80000000d350, buf_ptr = 0x80000000d410) = -0x2 (ENOENT)
[=]     openat(fd = 0xffffff9c, path = 0x80000000d350, flags = 0x80000, mode = 0x0) = -0x1 (EPERM)
[=]     stat(path = 0x80000000d350, buf_ptr = 0x80000000d410) = -0x2 (ENOENT)
[=]     openat(fd = 0xffffff9c, path = 0x80000000d350, flags = 0x80000, mode = 0x0) = -0x1 (EPERM)
[=]     stat(path = 0x80000000d350, buf_ptr = 0x80000000d410) = -0x2 (ENOENT)
[=]     openat(fd = 0xffffff9c, path = 0x80000000d350, flags = 0x80000, mode = 0x0) = -0x1 (EPERM)
[=]     stat(path = 0x80000000d350, buf_ptr = 0x80000000d410) = -0x2 (ENOENT)
[=]     openat(fd = 0xffffff9c, path = 0x80000000d350, flags = 0x80000, mode = 0x0) = -0x1 (EPERM)
[=]     stat(path = 0x80000000d350, buf_ptr = 0x80000000d410) = -0x2 (ENOENT)
[=]     openat(fd = 0xffffff9c, path = 0x80000000d350, flags = 0x80000, mode = 0x0) = -0x1 (EPERM)
[=]     stat(path = 0x80000000d350, buf_ptr = 0x80000000d410) = -0x2 (ENOENT)
[=]     openat(fd = 0xffffff9c, path = 0x80000000d350, flags = 0x80000, mode = 0x0) = -0x1 (EPERM)
[=]     stat(path = 0x80000000d350, buf_ptr = 0x80000000d410) = -0x2 (ENOENT)
[=]     openat(fd = 0xffffff9c, path = 0x80000000d350, flags = 0x80000, mode = 0x0) = -0x1 (EPERM)
[=]     stat(path = 0x80000000d350, buf_ptr = 0x80000000d410) = -0x2 (ENOENT)
[=]     openat(fd = 0xffffff9c, path = 0x80000000d350, flags = 0x80000, mode = 0x0) = -0x1 (EPERM)
[=]     stat(path = 0x80000000d350, buf_ptr = 0x80000000d410) = -0x2 (ENOENT)
[=]     openat(fd = 0xffffff9c, path = 0x80000000d350, flags = 0x80000, mode = 0x0) = -0x1 (EPERM)
[=]     stat(path = 0x80000000d350, buf_ptr = 0x80000000d410) = -0x2 (ENOENT)
[=]     openat(fd = 0xffffff9c, path = 0x80000000d350, flags = 0x80000, mode = 0x0) = -0x1 (EPERM)
[=]     stat(path = 0x80000000d350, buf_ptr = 0x80000000d410) = -0x2 (ENOENT)
[=]     openat(fd = 0xffffff9c, path = 0x80000000d350, flags = 0x80000, mode = 0x0) = -0x1 (EPERM)
[=]     stat(path = 0x80000000d350, buf_ptr = 0x80000000d410) = -0x2 (ENOENT)
[=]     openat(fd = 0xffffff9c, path = 0x80000000d350, flags = 0x80000, mode = 0x0) = -0x1 (EPERM)
[=]     stat(path = 0x80000000d350, buf_ptr = 0x80000000d410) = -0x2 (ENOENT)
[=]     openat(fd = 0xffffff9c, path = 0x80000000d350, flags = 0x80000, mode = 0x0) = -0x1 (EPERM)
[=]     stat(path = 0x80000000d350, buf_ptr = 0x80000000d410) = -0x2 (ENOENT)
[=]     openat(fd = 0xffffff9c, path = 0x80000000d350, flags = 0x80000, mode = 0x0) = -0x1 (EPERM)
[=]     stat(path = 0x80000000d350, buf_ptr = 0x80000000d410) = -0x2 (ENOENT)
[=]     openat(fd = 0xffffff9c, path = 0x80000000d350, flags = 0x80000, mode = 0x0) = -0x1 (EPERM)
[=]     stat(path = 0x80000000d350, buf_ptr = 0x80000000d410) = -0x2 (ENOENT)
[=]     openat(fd = 0xffffff9c, path = 0x80000000d350, flags = 0x80000, mode = 0x0) = -0x1 (EPERM)
[=]     stat(path = 0x80000000d350, buf_ptr = 0x80000000d410) = -0x2 (ENOENT)
[=]     openat(fd = 0xffffff9c, path = 0x80000000d350, flags = 0x80000, mode = 0x0) = -0x1 (EPERM)
[=]     stat(path = 0x80000000d350, buf_ptr = 0x80000000d410) = -0x2 (ENOENT)
[=]     openat(fd = 0xffffff9c, path = 0x80000000d350, flags = 0x80000, mode = 0x0) = -0x1 (EPERM)
[=]     stat(path = 0x80000000d350, buf_ptr = 0x80000000d410) = 0x0
/usr/bin/ls: error while loading shared libraries: libselinux.so.1: cannot open shared object file: Operation not permitted
[=]     writev(fd = 0x2, vec = 0x80000000d0c0, vlen = 0xa) = 0x7c
[=]     exit_group(code = 0x7f) = ?
[x]     CPU Context:
[x]     ah      : 0x0
[x]     al      : 0xe7
[x]     ch      : 0x0
[x]     cl      : 0x0
[x]     dh      : 0xd6
[x]     dl      : 0x0
[x]     bh      : 0x8c
[x]     bl      : 0xe0
[x]     ax      : 0xe7
[x]     cx      : 0x0
[x]     dx      : 0xd600
[x]     bx      : 0x8ce0
[x]     sp      : 0xd5e8
[x]     bp      : 0x0
[x]     si      : 0xa33
[x]     di      : 0x8730
[x]     ip      : 0x0
[x]     eax     : 0xe7
[x]     ecx     : 0x0
[x]     edx     : 0xd600
[x]     ebx     : 0xf7df8ce0
[x]     esp     : 0xd5e8
[x]     ebp     : 0x0
[x]     esi     : 0xf7df0a33
[x]     edi     : 0x55558730
[x]     eip     : 0x0
[x]     rax     : 0xe7
[x]     rbx     : 0x7ffff7df8ce0
[x]     rcx     : 0x0
[x]     rdx     : 0x80000000d600
[x]     rsi     : 0x7ffff7df0a33
[x]     rdi     : 0x555555558730
[x]     rbp     : 0x0
[x]     rsp     : 0x80000000d5e8
[x]     r8      : 0x0
[x]     r9      : 0x7f
[x]     r10     : 0x20
[x]     r11     : 0x0
[x]     r12     : 0x1
[x]     r13     : 0x7ffff7ffe960
[x]     r14     : 0x7ffff7ffe950
[x]     r15     : 0x7ffff7ffe170
[x]     rip     : 0x0
[x]     cr0     : 0x11
[x]     cr1     : 0x0
[x]     cr2     : 0x0
[x]     cr3     : 0x0
[x]     cr4     : 0x0
[x]     cr8     : 0x0
[x]     dr0     : 0x0
[x]     dr1     : 0x0
[x]     dr2     : 0x0
[x]     dr3     : 0x0
[x]     dr4     : 0x0
[x]     dr5     : 0x0
[x]     dr6     : 0xffff0ff0
[x]     dr7     : 0x400
[x]     st0     : 0x0
[x]     st1     : 0x0
[x]     st2     : 0x0
[x]     st3     : 0x0
[x]     st4     : 0x0
[x]     st5     : 0x0
[x]     st6     : 0x0
[x]     st7     : 0x0
[x]     eflags  : 0x46
[x]     cs      : 0x33
[x]     ss      : 0x28
[x]     ds      : 0x0
[x]     es      : 0x0
[x]     fs      : 0x0
[x]     gs      : 0x0
[x]     r8b     : 0x0
[x]     r9b     : 0x7f
[x]     r10b    : 0x20
[x]     r11b    : 0x0
[x]     r12b    : 0x1
[x]     r13b    : 0x60
[x]     r14b    : 0x50
[x]     r15b    : 0x70
[x]     r8w     : 0x0
[x]     r9w     : 0x7f
[x]     r10w    : 0x20
[x]     r11w    : 0x0
[x]     r12w    : 0x1
[x]     r13w    : 0xe960
[x]     r14w    : 0xe950
[x]     r15w    : 0xe170
[x]     r8d     : 0x0
[x]     r9d     : 0x7f
[x]     r10d    : 0x20
[x]     r11d    : 0x0
[x]     r12d    : 0x1
[x]     r13d    : 0xf7ffe960
[x]     r14d    : 0xf7ffe950
[x]     r15d    : 0xf7ffe170
[x]     fsbase  : 0x0
[x]     gsbase  : 0x6000000
[x]     xmm0    : 0x0
[x]     xmm1    : 0x0
[x]     xmm2    : 0xff0000000000
[x]     xmm3    : 0xff0000000000000000
[x]     xmm4    : 0x0
[x]     xmm5    : 0x0
[x]     xmm6    : 0x0
[x]     xmm7    : 0x0
[x]     xmm8    : 0x0
[x]     xmm9    : 0x0
[x]     xmm10   : 0x0
[x]     xmm11   : 0x0
[x]     xmm12   : 0x0
[x]     xmm13   : 0x0
[x]     xmm14   : 0x0
[x]     xmm15   : 0x0
[x]     xmm16   : 0x0
[x]     xmm17   : 0x0
[x]     xmm18   : 0x0
[x]     xmm19   : 0x0
[x]     xmm20   : 0x0
[x]     xmm21   : 0x0
[x]     xmm22   : 0x0
[x]     xmm23   : 0x0
[x]     xmm24   : 0x0
[x]     xmm25   : 0x0
[x]     xmm26   : 0x0
[x]     xmm27   : 0x0
[x]     xmm28   : 0x0
[x]     xmm29   : 0x0
[x]     xmm30   : 0x0
[x]     xmm31   : 0x0
[x]     ymm0    : 0x0
[x]     ymm1    : 0x0
[x]     ymm2    : 0xff0000000000
[x]     ymm3    : 0xff0000000000000000
[x]     ymm4    : 0x0
[x]     ymm5    : 0x0
[x]     ymm6    : 0x0
[x]     ymm7    : 0x0
[x]     ymm8    : 0x0
[x]     ymm9    : 0x0
[x]     ymm10   : 0x0
[x]     ymm11   : 0x0
[x]     ymm12   : 0x0
[x]     ymm13   : 0x0
[x]     ymm14   : 0x0
[x]     ymm15   : 0x0
[x]     ymm16   : 0x0
[x]     ymm17   : 0x0
[x]     ymm18   : 0x0
[x]     ymm19   : 0x0
[x]     ymm20   : 0x0
[x]     ymm21   : 0x0
[x]     ymm22   : 0x0
[x]     ymm23   : 0x0
[x]     ymm24   : 0x0
[x]     ymm25   : 0x0
[x]     ymm26   : 0x0
[x]     ymm27   : 0x0
[x]     ymm28   : 0x0
[x]     ymm29   : 0x0
[x]     ymm30   : 0x0
[x]     ymm31   : 0x0
[x]     zmm0    : 0x0
[x]     zmm1    : 0x0
[x]     zmm2    : 0x0
[x]     zmm3    : 0x0
[x]     zmm4    : 0x0
[x]     zmm5    : 0x0
[x]     zmm6    : 0x0
[x]     zmm7    : 0x0
[x]     zmm8    : 0x0
[x]     zmm9    : 0x0
[x]     zmm10   : 0x0
[x]     zmm11   : 0x0
[x]     zmm12   : 0x0
[x]     zmm13   : 0x0
[x]     zmm14   : 0x0
[x]     zmm15   : 0x0
[x]     zmm16   : 0x0
[x]     zmm17   : 0x0
[x]     zmm18   : 0x0
[x]     zmm19   : 0x0
[x]     zmm20   : 0x0
[x]     zmm21   : 0x0
[x]     zmm22   : 0x0
[x]     zmm23   : 0x0
[x]     zmm24   : 0x0
[x]     zmm25   : 0x0
[x]     zmm26   : 0x0
[x]     zmm27   : 0x0
[x]     zmm28   : 0x0
[x]     zmm29   : 0x0
[x]     zmm30   : 0x0
[x]     zmm31   : 0x0
[x]     PC = 0x0000000000000000 (unreachable)

[x]     Memory map:
[x]     Start                End                  Perm    Label                  Image
[x]     000000000000030000 - 000000000000031000   rwx     [GDT]                  
[x]     000000555555554000 - 000000555555558000   r--     ls                     /usr/bin/ls
[x]     000000555555558000 - 00000055555556e000   r-x     ls                     /usr/bin/ls
[x]     00000055555556e000 - 000000555555577000   r--     ls                     /usr/bin/ls
[x]     000000555555577000 - 00000055555557a000   rw-     ls                     /usr/bin/ls
[x]     00000055555557a000 - 00000055555557c000   rwx     [hook_mem]             
[x]     0000007ffff7dd5000 - 0000007ffff7dfc000   r-x     ld-linux-x86-64.so.2   /home/kali/Desktop/404_RE/qiling/examples/rootfs/x8664_linux/lib64/ld-linux-x86-64.so.2
[x]     0000007ffff7ffc000 - 0000007ffff7fff000   rw-     ld-linux-x86-64.so.2   /home/kali/Desktop/404_RE/qiling/examples/rootfs/x8664_linux/lib64/ld-linux-x86-64.so.2
[x]     0000007ffffffde000 - 00000080000000e000   rwx     [stack]                
[x]     00ffffffffff600000 - 00ffffffffff601000   rwx     [vsyscall]             
Traceback (most recent call last):
  File "/home/kali/Desktop/404_RE/disas.py", line 10, in <module>
    disas()
  File "/home/kali/Desktop/404_RE/disas.py", line 7, in disas
    ql.run()
  File "/home/kali/Desktop/404_RE/.venv/lib/python3.10/site-packages/qiling/core.py", line 600, in run
    self.os.run()
  File "/home/kali/Desktop/404_RE/.venv/lib/python3.10/site-packages/qiling/os/linux/linux.py", line 184, in run
    self.ql.emu_start(self.ql.loader.elf_entry, self.exit_point, self.ql.timeout, self.ql.count)
  File "/home/kali/Desktop/404_RE/.venv/lib/python3.10/site-packages/qiling/core.py", line 774, in emu_start
    self.uc.emu_start(begin, end, timeout, count)
  File "/home/kali/Desktop/404_RE/.venv/lib/python3.10/site-packages/unicorn/unicorn.py", line 547, in emu_start
    raise UcError(status)
unicorn.unicorn.UcError: Invalid memory fetch (UC_ERR_FETCH_UNMAPPED)
elicn commented 1 year ago

This is most likely caused due to a mismatch between the binary and the rootfs that is used to load it. You can try using '/' as rootfs and see if it works. Normally using your hosting system root directory is not recommended, but this should be OK if you can trust the binary you are emulating and know it is harmless.

NudistBeaaach commented 1 year ago

This is most likely caused due to a mismatch between the binary and the rootfs that is used to load it. You can try using '/' as rootfs and see if it works. Normally using your hosting system root directory is not recommended, but this should be OK if you can trust the binary you are emulating and know it is harmless.

I set my '/' as the rootfs and now I got the CPU ISA is lower than required that i previously saw in another issue, I added this snippet to tackle the problem but nop, same error CPU ISA is lower than required 

    def null_rseq_impl(ql: Qiling, abi: int, length: int, flags: int, sig: int):
        # do nothing
        pass

    ql.os.set_syscall('rseq', null_rseq_impl, QL_INTERCEPT.CALL)
elicn commented 1 year ago

I believe this one is caused due to arch_prctl being used with ARCH_CET_STATUS. You can work around it by returning an error that will indicate that the feature is not supported:

from qiling.const import QL_INTERCEPT

def __no_cet(ql: Qiling, code: int, addr: int, retval: int):
    # intercept arch_prctl syscall after it exits. if code was set to ARCH_CET_STATUS,
    # then return an error value. otherwise, return the value originally returned by
    # the syscall.
    #
    # note: if -1 doesn't work, maybe should use -22 (-EINVAL) instead
    return -1 if code == 0x3001 else retval

ql.os.set_syscall('arch_prctl', __no_cet, QL_INTERCEPT.EXIT)
NudistBeaaach commented 1 year ago

I tried on a 2.35 libc, and the patch I found in previous issues and here: https://cloud.tencent.com/developer/article/2144036 worked well, I still get a warning for the prctl syscall but it's running with no crash. But on newer libc the patchs are not working probably due to the hardcoded offsets which are maybe not the sames. Btw with this latest snippet of code I still get the warning [!] prctl code 0x3001 not implemented, I don't know if removing it is possible but thx anyway for you help and quick support.

elicn commented 1 year ago

The snippet above just patches the returned value, so you'll still be seeing the warning message. I guess we can fix that though, should be a straightforward one.

Newer libc on Intel-based systems attempt to enable Intel CET, if it is supported. The problem with prctl is that it reports "OK" for everything (return value 0), even if it is not actually supported. Based on that libc tries to use CET and fail because the default CPU emulated by Unicorn, under the hood, does not support it.

The long-term fix will be allowing users to select the emulated CPU (say, Icelake, or equivalent) so the feature will be supported. The short-term fix is just let libc know the feature is not supported, and that is what the snippet above does: it patches the returned value to a "not supported" value when the system is queried about CET.