Support Qiling debug

[redmed666]

Describe the solution you'd like

I think supporting debugging with Qiling framework would be a pretty cool feature to have. Indeed, Qiling can fill the gap between light emulation (covered by radare2) and complete executable running.

Describe alternatives you've considered

I tried to connect to Qiling gdb server but it didn't work as expected. I think the particular way to make debugging within radare2 working in this case is the issue.

Additional context

In order to debug correctly an executable by connecting to the qiling gdb server:

1) r2 - 2) doof gdb://<ip>:<port>/0

Analyzing the executable before running doof doesn't work (which is weird for me).

[ITAYC0HEN]

Hey @redmed666 ! Thanks for opening this issue :) Indded, Qiling looks great and when working with Cutter it will be terrific! We tested radare2 with Qiling several times, and sadly there are issues in Qiling side that is blocking this from work. Specifically, it doesn't support some of the XML fields needed for the communciation protocol with gdb. We will check better and let the nice folks at Qiling team know about the gaps that needed to be solved.

POC: https://twitter.com/trufae/status/1254320307688611841

@yossizap will know to tell more about this

[redmed666]

Hi @ITAYC0HEN!

Thank you for your answer. I hope it will be fixed.

[xwings]

heh, i see this post cannot be unseen.

how can we fix this and get this moving.

[XVilka]

@abcSup can also take a look if @yossizap is busy.

[xwings]

we just tag 1.1-alpha1, will you guys able to check it ?

pip3 install qiling --pre

this will do

[abcSup]

we just tag 1.1-alpha1, will you guys able to check it ?

I will look into it tonight and thanks for the installation tips.

[redmed666]

Unfortunately, with the latest version from github, I have this error message:

debugger> Error: Not able to initialize Debugging Server
Traceback (most recent call last):
  File "./qltool", line 261, in <module>
    ql.run(timeout=timeout)
  File "/root/qiling/qiling/core.py", line 191, in run
    ql_debugger_init(self)
  File "/root/qiling/qiling/debugger/debugger.py", line 78, in ql_debugger_init
    ql_debugger(ql, remotedebugsrv, ip, port)
  File "/root/qiling/qiling/debugger/debugger.py", line 47, in ql_debugger
    ql.remote_debug = DEBUGSESSION(ql, conn, exit_point, mappings)
  File "/root/qiling/qiling/debugger/gdbserver/gdbserver.py", line 45, in __init__
    self.gdb.initialize(self.ql, exit_point=exit_point, mappings=mappings)
  File "/root/qiling/qiling/debugger/gdbserver/qldbg.py", line 27, in initialize
    self.current_address = self.entry_point = self.ql.os.entry_point
AttributeError: 'QlOsWindows' object has no attribute 'entry_point'

[xwings]

screw me. i forgotten that fix. can u clone and try dev now?

[xwings]

u r right. i am looking at it. i will fix in dev. for pip need to wait till rc2

[abcSup]

Unfortunately, with the latest version from github, I have this error message:

debugger> Error: Not able to initialize Debugging Server
Traceback (most recent call last):
  File "./qltool", line 261, in <module>
    ql.run(timeout=timeout)
  File "/root/qiling/qiling/core.py", line 191, in run
    ql_debugger_init(self)
  File "/root/qiling/qiling/debugger/debugger.py", line 78, in ql_debugger_init
    ql_debugger(ql, remotedebugsrv, ip, port)
  File "/root/qiling/qiling/debugger/debugger.py", line 47, in ql_debugger
    ql.remote_debug = DEBUGSESSION(ql, conn, exit_point, mappings)
  File "/root/qiling/qiling/debugger/gdbserver/gdbserver.py", line 45, in __init__
    self.gdb.initialize(self.ql, exit_point=exit_point, mappings=mappings)
  File "/root/qiling/qiling/debugger/gdbserver/qldbg.py", line 27, in initialize
    self.current_address = self.entry_point = self.ql.os.entry_point
AttributeError: 'QlOsWindows' object has no attribute 'entry_point'

Could you provide me the executable you running? I believe the issue above is related to qiling.

I have installed qiling 1.1-alpha1 and tried the remote debugging. I can confirm that it works for the example using qltool using the latest r2:

$ ./qltool 
qltool for Qiling v1.1-alpha1

Usage: ./qltool [run|shellcode] OPTIONS
----------snipped----------
$ r2 -v
radare2 4.5.0-git 24812 @ linux-x86-64 git.4.4.0-212-gc9eceab2d
commit: c9eceab2d653462dd9c1be7a76a83555e7068bde build: 2020-05-25__23:48:27

$ ./qltool run -f examples/rootfs/x8664_linux/bin/x8664_hello --gdb 127.0.0.1:9999 --rootfs examples/rootfs/x8664_linux
debugger> Initializing load_address 0x555555554000
debugger> Listening on 127.0.0.1:9999

$ r2 -d gdb://localhost:9999
= attach 0 1
= attach 0 0
= attach 0 0
= attach 0 0
 -- Try with ASAN, and be amazed
[0x555555554530]> dr
rax = 0x0000001c
rbx = 0x00000000
rcx = 0x00000004
rdx = 0x7ffff7de59a0
rsi = 0x7ffff7ffe700
rdi = 0x00000000
rbp = 0x00000000
rsp = 0x80000000de80
r8 = 0x8000000a
r9 = 0x00000000
r10 = 0x7ffff7ffd9f0
r11 = 0x7ffff7ffe930
r12 = 0x555555554530
r13 = 0x80000000de80
r14 = 0x00000000
r15 = 0x00000000
rip = 0x555555554530
orig_rax = 0x00000000
fs_base = 0x00000000
gs_base = 0x00000000
bndcfgu = 0x00000000
bndstatus = 0x00000000
[0x555555554530]> pd 5
            ;-- r12:
            ;-- rip:
            0x555555554530      31ed           xor ebp, ebp
            0x555555554532      4989d1         mov r9, rdx
            0x555555554535      5e             pop rsi
            0x555555554536      4889e2         mov rdx, rsp
            0x555555554539      4883e4f0       and rsp, 0xfffffffffffffff0
[0x555555554530]> 

Please upload the example you are trying to debug, so I can try to reproduce the problem and help you out anytime.

[xwings]

you can clone the latest qiling from

git clone git@github.com:qilingframework/qiling.git
cd qiling
git checkout dev

and run

./qltool run -f examples/rootfs/x8664_linux/bin/x8664_hello --gdb 127.0.0.1:9999 --rootfs examples/rootfs/x8664_linux

should be good now

[redmed666]

Indeed, this issue is now resolved but I have another one

./qltool run -f ./examples/rootfs/x8664_windows/bin/x8664_hello.exe --rootfs /mnt/hgfs/Downloads/rootfs/ --gdb 0.0.0.0:9998
[+] Initiate stack address at 0x7ffffffde000
[+] Loading ./examples/rootfs/x8664_windows/bin/x8664_hello.exe to 0x400000
[+] PE entry point at 0x4014e0
[+] TEB addr is 0x6030
[+] PEB addr is 0x60b8
[+] Loading /mnt/hgfs/Downloads/rootfs/Windows/System32/kernel32.dll to 0x7ffff0000000
[+] Done with loading /mnt/hgfs/Downloads/rootfs/Windows/System32/kernel32.dll
[+] Loading /mnt/hgfs/Downloads/rootfs/Windows/System32/msvcrt.dll to 0x7ffff00b2000
[+] Done with loading /mnt/hgfs/Downloads/rootfs/Windows/System32/msvcrt.dll
debugger> Initializing load_address 0x0
debugger> Listening on 0.0.0.0:9998
gdb> Breakpoint added at: 0x4014e0
gdb> Breakpoint found, stop at address: 0x4014e0
gdb> Xml file not found: /root/qiling/qiling/debugger/gdbserver/xml/x8664/target.xml

which is weird because the file exists...

l /root/qiling/qiling/debugger/gdbserver/xml/x8664/target.xml
-rw-r--r-- 1 root root 356 May 10 09:36 /root/qiling/qiling/debugger/gdbserver/xml/x8664/target.xml

Thank you for your help and your responses!

[xwings]

this is some elf related issue. but this will not effect r2 i guess.

maybe is should make a check if this is a *nix. let me try

[xwings]

i added a better response msg. It should work now. Will you be able to try again ?

[redmed666]

It's indeed clearer like that. Thanks!

It's still doesn't work for me. I have a segfault on the radare2 side. Do you have the same thing with a Windows executable?

[abcSup]

I will try it out, but I have a question about running a Windows executable on qiling. Do I need an installation of Windows OS to use it as my rootfs?

[abcSup]

I will try it out, but I have a question about running a Windows executable on qiling. Do I need an installation of Windows OS to use it as my rootfs?

It is ok. I found the official guide regarding Windows emulation and will try it out now. https://www.qiling.io/install/

[redmed666]

I populated my rootfs with the DLLs of my Windows 10 VM. But I think you should be able to run the examples with the DLLs from ReactOS. Here is a folder structure example

l /mnt/hgfs/Downloads/rootfs/Windows
drwxr-xr-x 1 501 dialout  192 May 26 09:23 .
drwxr-xr-x 1 501 dialout  160 Mar  4 21:50 ..
drwxr-xr-x 1 501 dialout  192 Mar  4 21:48 registry
drwxr-xr-x 1 501 dialout 106K Mar  4 21:24 System32
drwxr-xr-x 1 501 dialout  73K May 14 13:37 SysWOW64

[abcSup]

I populated my rootfs with the DLLs of my Windows 10 VM. But I think you should be able to run the examples with the DLLs from ReactOS. Here is a folder structure example

l /mnt/hgfs/Downloads/rootfs/Windows
drwxr-xr-x 1 501 dialout  192 May 26 09:23 .
drwxr-xr-x 1 501 dialout  160 Mar  4 21:50 ..
drwxr-xr-x 1 501 dialout  192 Mar  4 21:48 registry
drwxr-xr-x 1 501 dialout 106K Mar  4 21:24 System32
drwxr-xr-x 1 501 dialout  73K May 14 13:37 SysWOW64

Thank you!

[xwings]

It's indeed clearer like that. Thanks!

It's still doesn't work for me. I have a segfault on the radare2 side. Do you have the same thing with a Windows executable?

i am using x86_helloworld which comes with the example in qiling. u can get it from https://github.com/qilingframework/qiling , give us a star too. on our way to 1,000 start :)

[xwings]

I populated my rootfs with the DLLs of my Windows 10 VM. But I think you should be able to run the examples with the DLLs from ReactOS. Here is a folder structure example

l /mnt/hgfs/Downloads/rootfs/Windows
drwxr-xr-x 1 501 dialout  192 May 26 09:23 .
drwxr-xr-x 1 501 dialout  160 Mar  4 21:50 ..
drwxr-xr-x 1 501 dialout  192 Mar  4 21:48 registry
drwxr-xr-x 1 501 dialout 106K Mar  4 21:24 System32
drwxr-xr-x 1 501 dialout  73K May 14 13:37 SysWOW64

Thank you!

u can raise a issue in qiling. dont pollute r2's github :)

[redmed666]

It's indeed clearer like that. Thanks! It's still doesn't work for me. I have a segfault on the radare2 side. Do you have the same thing with a Windows executable?

i am using x86_helloworld which comes with the example in qiling. u can get it from https://github.com/qilingframework/qiling , give us a star too. on our way to 1,000 start :)

Don't worry, it's already done :D

[redmed666]

I populated my rootfs with the DLLs of my Windows 10 VM. But I think you should be able to run the examples with the DLLs from ReactOS. Here is a folder structure example

l /mnt/hgfs/Downloads/rootfs/Windows
drwxr-xr-x 1 501 dialout  192 May 26 09:23 .
drwxr-xr-x 1 501 dialout  160 Mar  4 21:50 ..
drwxr-xr-x 1 501 dialout  192 Mar  4 21:48 registry
drwxr-xr-x 1 501 dialout 106K Mar  4 21:24 System32
drwxr-xr-x 1 501 dialout  73K May 14 13:37 SysWOW64

Thank you!

u can raise a issue in qiling. dont pollute r2's github :)

Alright, will do. Thank you for your time and your quick answers!

[xwings]

Ok, we manage to fix r2 and cutter is not working with gdb remote debugging.

I guess we can come back here now.

[abcSup]

It's indeed clearer like that. Thanks!

It's still doesn't work for me. I have a segfault on the radare2 side. Do you have the same thing with a Windows executable?

@redmed666 I am able to reproduce the segfault case when running a windows exe. It seems to be some memory leak issues in r2.

gdb> Breakpoint added at: 0x4014e0
gdb> Breakpoint found, stop at address: 0x4014e0
gdb> Platform is not supported by xml or xml file not found: /qiling/debugger/gdbserver/xml/x8664/target.xml

this is some elf related issue. but this will not effect r2 i guess.

@xwings You are right that it will not effect r2. r2 will use a generic register profile when provided no target.xml.

[xwings]

I do believe we should work on a closer integration with r2 and lets not reply on this GDB RSP thingy

[abcSup]

I do believe we should work on a closer integration with r2 and lets not reply on this GDB RSP thingy

I will resolve this issue first. Please open a new PR and discuss about features you wish to integrate with r2 and we can work from there. Thanks!

[xwings]

its more on PR on Qiling. I am not too sure r2 have any r2's protocol or we just need to work it out on rsp

if r2 already got a remote debugging protocol then it will my issue on my side. :)

[abcSup]

r2 does not have any its own remote debugging protocol

. I would say using GDB RSP is good for now because we do not have to develop new protocol ourselves, and r2 and qiling are already supporting GDB RSP.

[4rchib4ld]

Hello ! Since Cutter has a beta feature for a remote debugger, is there now a way to make it work with Qiling ? Would be great !