search

radareorg / radare2

UNIX-like reverse engineering framework and command-line toolset
https://www.radare.org/
GNU Lesser General Public License v3.0
20.7k stars 3.01k forks source link

Decoding QNX Executable file #12664

Closed karthick-gururaj closed 5 years ago

karthick-gururaj commented 5 years ago

Work environment

Questions Answers
OS/arch/bits Windows 10 / x86 / 64
File format of the file being disassembled QNX Executable format
Architecture/bits of the file x86/32 (386)
r2 -v full output radare2 3.1.3 6 @ windows-x86-32 git.3.1.3-5-gdf4c9f0 commit: df4c9f09ba6fe07a30b55a5c40c5bbdb7bddf087 build: Tue 12/04/2018__21:40:59.09

Expected behavior

radare2 should decode the format to figure out where code blocks are and then disassemble the code blocks. The attached zip has reference disassembly files.

Actual behavior

Disassembly starts from the very first byte - possibly because the format is not recognized.

Hexdump of first few bytes of the executable file header is (question marks indicate the bytes that differ between 3 executables that were inspected): 

00000000: 0000 3800 0000 9001 1600 8201 0000 0000  ..8............. 
00000010: 0100 0100 0100 0000 0000 0000 0000 ????  ..............?? 
00000020: 0000 0080 0000 0010 0000 00a0 0000 0000  ................ 
00000030: 0000 0000 0000 ???? 0020 ???? 0000 ????  ......??. ??..??

The hexdump of first few bytes of the actual executable being disassembled:

00000000: 0000 3800 0000 9001 1600 8201 0000 0000  ..8.............
00000010: 0100 0100 0100 0000 0000 0000 0000 1c02  ................
00000020: 0000 0080 0000 0010 0000 00a0 0000 0000  ................
00000030: 0000 0000 0000 c306 0020 1001 0000 0200  ......... ......

The disassembly produced by radare2 is (for debug/test.x in attached zip):

0x00000000      0000           add byte [eax], al
0x00000002      3800           cmp byte [eax], al
0x00000004      0000           add byte [eax], al
0x00000006      90             nop
0x00000007      0116           add dword [esi], edx
0x00000009      008201000000   add byte [edx + 1], al
0x0000000f      0001           add byte [ecx], al
0x00000011      0001           add byte [ecx], al
0x00000013      0001           add byte [ecx], al
0x00000015      0000           add byte [eax], al
0x00000017      0000           add byte [eax], al
0x00000019      0000           add byte [eax], al
0x0000001b      0000           add byte [eax], al
0x0000001d      001c02         add byte [edx + eax], bl
0x00000020      0000           add byte [eax], al
0x00000022      008000000010   add byte [eax + 0x10000000], al
0x00000028      0000           add byte [eax], al
0x0000002a      00a000000000   add byte [eax], ah
0x00000030      0000           add byte [eax], al
0x00000032      0000           add byte [eax], al
0x00000034      0000           add byte [eax], al
0x00000036      c3             ret
0x00000037      06             push es
0x00000038      0020           add byte [eax], ah
0x0000003a      1001           adc byte [ecx], al
0x0000003c      0000           add byte [eax], al
0x0000003e      0200           add al, byte [eax]
0x00000040      fc             cld
0x00000041      0000           add byte [eax], al

Steps to reproduce the behavior

The attached zip has the following files:

Compare the reference disassembly to what is produced by radare2:

cmd> radare2 test.x
[0x00000000]> pd $r
radare commented 5 years ago

no, this binary format is not supported by r2's RBin yet. do y ou have documentation about it?

All the QNX bins ive seen before were basically ELFs.. what's the name of this executable format? can you provide magic signatures for them too?

On 5 Jan 2019, at 15:18, karthick-gururaj notifications@github.com wrote:

Work environment

Questions Answers OS/arch/bits Windows 10 / x86 / 64 File format of the file being disassembled QNX Executable format Architecture/bits of the file x86/32 (386) r2 -v full output radare2 3.1.3 6 @ windows-x86-32 git.3.1.3-5-gdf4c9f0 commit: df4c9f0 https://github.com/radare/radare2/commit/df4c9f09ba6fe07a30b55a5c40c5bbdb7bddf087 build: Tue 12/04/2018__21:40:59.09 Expected behavior

radare2 should decode the format to figure out where code blocks are and then disassemble the code blocks. The attached zip https://github.com/radare/radare2/files/2729570/simple_test.zip has reference disassembly files.

Actual behavior

Disassembly starts from the very first byte - possibly because the format is not recognized.

Hexdump of first few bytes of the executable file header is (question marks indicate the bytes that differ between 3 executables that were inspected):

00000000: 0000 3800 0000 9001 1600 8201 0000 0000 ..8............. 00000010: 0100 0100 0100 0000 0000 0000 0000 ???? ..............?? 00000020: 0000 0080 0000 0010 0000 00a0 0000 0000 ................ 00000030: 0000 0000 0000 ???? 0020 ???? 0000 ???? ......??. ??..?? The hexdump of first few bytes of the actual executable being disassembled:

00000000: 0000 3800 0000 9001 1600 8201 0000 0000 ..8............. 00000010: 0100 0100 0100 0000 0000 0000 0000 1c02 ................ 00000020: 0000 0080 0000 0010 0000 00a0 0000 0000 ................ 00000030: 0000 0000 0000 c306 0020 1001 0000 0200 ......... ...... The disassembly produced by radare2 is (for debug/test.x in attached zip https://github.com/radare/radare2/files/2729570/simple_test.zip):

0x00000000 0000 add byte [eax], al 0x00000002 3800 cmp byte [eax], al 0x00000004 0000 add byte [eax], al 0x00000006 90 nop 0x00000007 0116 add dword [esi], edx 0x00000009 008201000000 add byte [edx + 1], al 0x0000000f 0001 add byte [ecx], al 0x00000011 0001 add byte [ecx], al 0x00000013 0001 add byte [ecx], al 0x00000015 0000 add byte [eax], al 0x00000017 0000 add byte [eax], al 0x00000019 0000 add byte [eax], al 0x0000001b 0000 add byte [eax], al 0x0000001d 001c02 add byte [edx + eax], bl 0x00000020 0000 add byte [eax], al 0x00000022 008000000010 add byte [eax + 0x10000000], al 0x00000028 0000 add byte [eax], al 0x0000002a 00a000000000 add byte [eax], ah 0x00000030 0000 add byte [eax], al 0x00000032 0000 add byte [eax], al 0x00000034 0000 add byte [eax], al 0x00000036 c3 ret 0x00000037 06 push es 0x00000038 0020 add byte [eax], ah 0x0000003a 1001 adc byte [ecx], al 0x0000003c 0000 add byte [eax], al 0x0000003e 0200 add al, byte [eax] 0x00000040 fc cld 0x00000041 0000 add byte [eax], al Steps to reproduce the behavior

The attached zip https://github.com/radare/radare2/files/2729570/simple_test.zip has the following files:

test.c -> the C source file I'm using to compile debug/ contains objects compiled in debug mode (with -g option) and no_debug/ contains objects compiled without -g option Inside debug and no_debug folders, test.o -> object file, test.x -> final executable. Disassembly (test.S) and linker map (test.x.map) files are also included for reference. Compare the reference disassembly to what is produced by radare2:

cmd> radare2 test.x [0x00000000]> pd $r — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/radare/radare2/issues/12664, or mute the thread https://github.com/notifications/unsubscribe-auth/AA3-lgoHs0-P5OW97H29WCWHMK4zLCGzks5vALQ1gaJpZM4ZxHqv.

karthick-gururaj commented 5 years ago

The format is referred to as "QNX Executable format" and is not ELF. I had posted a question on the Open Watcom compiler group (the tool chain I'm using) on understanding the format better and got few hints. Documentation is limited - I found just one that had some potentially useful information - http://watcom.markoverholser.com/manuals/current/cguide.pdf (the parent folder has other tool chain manuals)

The format is as per the header file bld/watcom/h/exeqnx.h:

The file is a list of records of type lmf_record. The very first record is decoded from the first 6 bytes (which can be treated as the magic signature): 

0x00     -> lmf_record::rec_type (0 means LMF_HEADER_REC)
0x00     -> lmf_record::reserved
0x0038   -> lmf_record::data_nbytes (size of data record that follows, lmf_header in this case)
0000     -> lmf_record::spare

The next 0x0038 (56) bytes are of type lmf_header:

0x0190       -> lmf_header::version
0x0016       -> lmf_header::cflags
0x0182       -> lmf_header::cpu (can be: 86,186,286,386,486 etc. 0x0182 = 386 decimal)
0x0000       -> lmf_header::fpu (can be: 0, 87,287,387)
0x0000       -> lmf_header::code_index  (segment of code start)
0x0001       -> lmf_header::stack_index (segment to put the stack)
0x0001       -> lmf_header::heap_index  (segment to start DS at.)
0x0001       -> lmf_header::argv_index  (segment to put argv & environment.)
0x0000       -> lmf_header::spare2[0] (must be zero)
0x0000       -> lmf_header::spare2[1] (must be zero)
0x0000       -> lmf_header::spare2[2] (must be zero)
0x0000       -> lmf_header::spare2[3] (must be zero)
0x0000021c   -> lmf_header::code_offset (starting offset of code.)
0x00008000   -> lmf_header::stack_nbytes (stack size)
0x00001000   -> lmf_header::heap_nbytes (initial size of heap (optional).)
0x0000a000   -> lmf_header::image_base (starting address of image)
0x00000000   -> lmf_header::spare3[0]
0x00000000   -> lmf_header::spare3[1]

Does this information help?

radare commented 5 years ago

Thanks that's enough tos tart! is there any other tool that supports this file format ? maybe a fork of binutils?

Any other resource is welcome, also do you plan to contribute to this or it is fine for you to wait until someone else gets some spare time to care about it?

--pancake

On 5 Jan 2019, at 16:38, karthick-gururaj notifications@github.com wrote:

The format is referred to as "QNX Executable format" and is not ELF. I had posted a question on the Open Watcom compiler group https://groups.google.com/forum/#!topic/openwatcom.contributors/ipm6pi92e14 (the tool chain I'm using) on understanding the format better and got few hints. Documentation is limited - I found just one that had some potentially useful information - http://watcom.markoverholser.com/manuals/current/cguide.pdf http://watcom.markoverholser.com/manuals/current/cguide.pdf (the parent http://watcom.markoverholser.com/manuals/current folder has other tool chain manuals)

The format is as per the header file bld/watcom/h/exeqnx.h https://github.com/open-watcom/open-watcom-v2/blob/master/bld/watcom/h/exeqnx.h:

The file is a list of records of type lmf_record https://github.com/open-watcom/open-watcom-v2/blob/master/bld/watcom/h/exeqnx.h#L62. The very first record is decoded from the first 6 bytes (which can be treated as the magic signature):

0x00 -> lmf_record::rec_type (0 means LMF_HEADER_REC) 0x00 -> lmf_record::reserved 0x0038 -> lmf_record::data_nbytes (size of data record that follows, lmf_header in this case) 0000 -> lmf_record::spare The next 0x0038 (56) bytes are of type lmf_header:

0x0190 -> lmf_header::version 0x0016 -> lmf_header::cflags 0x0182 -> lmf_header::cpu (can be: 86,186,286,386,486 etc. 0x0182 = 386 decimal) 0x0000 -> lmf_header::fpu (can be: 0, 87,287,387) 0x0000 -> lmf_header::code_index (segment of code start) 0x0001 -> lmf_header::stack_index (segment to put the stack) 0x0001 -> lmf_header::heap_index (segment to start DS at.) 0x0001 -> lmf_header::argv_index (segment to put argv & environment.) 0x0000 -> lmf_header::spare2[0] (must be zero) 0x0000 -> lmf_header::spare2[1] (must be zero) 0x0000 -> lmf_header::spare2[2] (must be zero) 0x0000 -> lmf_header::spare2[3] (must be zero) 0x0000021c -> lmf_header::code_offset (starting offset of code.) 0x00008000 -> lmf_header::stack_nbytes (stack size) 0x00001000 -> lmf_header::heap_nbytes (initial size of heap (optional).) 0x0000a000 -> lmf_header::image_base (starting address of image) 0x00000000 -> lmf_header::spare3[0] 0x00000000 -> lmf_header::spare3[1] Does this information help?

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/radare/radare2/issues/12664#issuecomment-451665415, or mute the thread https://github.com/notifications/unsubscribe-auth/AA3-lvfsfkPXGRpjlEdtJqYHM8y32h-7ks5vAMbogaJpZM4ZxHqv.

karthick-gururaj commented 5 years ago

I'm not aware of any other tool that supports this format other than Watcom toolchain.

I'm not sure if I'll be able to contribute significant effort right now :( But if someone else does pick this up, I can pitch in. Can you point me someplace in code that I can look at to see how to add support for a new file format? I'll pursue that at my own pace, and see how far I can proceed.

Radare2 has one of the nerdiest interfaces I have seen in recent times - thanks for all the good work!

Maijin commented 5 years ago

See https://radare.gitbooks.io/radare2book/content/plugins/dev-bin.html

radare commented 5 years ago

I will try to find some time to implement that. Shouldnt take more than an hour. But full complete support will require more time and samples.

On 6 Jan 2019, at 13:12, Maijin notifications@github.com wrote:

See https://radare.gitbooks.io/radare2book/content/plugins/dev-bin.html

— You are receiving this because you commented. Reply to this email directly, view it on GitHub, or mute the thread.

karthick-gururaj commented 5 years ago

Thanks! I'll test the changes and feedback.

I'm noting down a one more source that I studied - that could be helpful.

wdump is a tool that dumps meta-data of the QNX executable. Output of running wdump on the debug/test.x is as below:

Open Watcom Executable Image Dump Utility Version 1.9
Portions Copyright (c) 1984-2002 Sybase, Inc. All Rights Reserved.
Source code is available under the Sybase Open Watcom Public License.
See http://www.openwatcom.org/ for details.

                                QNX EXE Header
==============================================================================
version number                              =     0190H
cflags                                      =     0016H
cpu (86, 186, 286, 386, 486)                =     0182H
fpu (0, 87, 287, 287)                       =     0000H
selector of code start address              =     0000H
selector where to put the stack             =     0001H
selector to put heap                        =     0001H
selector where to put argv and environment  =     0001H
zero                                        =     0000H
zero                                        =     0000H
zero                                        =     0000H
zero                                        =     0000H
code offset                                 = 0000021CH
stack size in bytes                         = 00008000H
initial near heap size in bytes             = 00001000H
image base                                  = 0000A000H
zero                                        = 00000000H
zero                                        = 00000000H
flags                                       = 32BIT PRIV_MASK FLAT
segments = type:size
type  0==read/write, 1==read-only, 2==execute/read, 3==execute-only
2:00006C3    0:0000110

                                 Data Table 1
==============================================================================
segment index          =     0001H
offset                 = 00000000H
size                   =     00F6H

                                Read/Write end
==============================================================================
verify = D5F5H     signature = A079CBDDH

                                 Data Table 2
==============================================================================
segment index          =     0000H
offset                 = 00000000H
size                   =     06C3H

The source file of wdump is useful reference, the function Dmp_qnx_head is the entry point.

There is also a loader bld/wl/c/loadqnx.c - which could also be helpful.

deepakchethan commented 5 years ago

Hey, I could start working on this if no one else is. It will take some time but I'll do it as fast as I can.

radare commented 5 years ago

Sure :) no hurry, feel free to ask anything you need

On 14 Feb 2019, at 01:58, Deepak Chethan notifications@github.com wrote:

Hey, I could start working on this if no one else is. It will take some time but I'll do it as fast as I can.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/radare/radare2/issues/12664#issuecomment-463440290, or mute the thread https://github.com/notifications/unsubscribe-auth/AA3-ls1KU8yemzamJfPhBser8a4f1yN-ks5vNLTCgaJpZM4ZxHqv.

deepakchethan commented 5 years ago

@karthick-gururaj can you please check and tell me if I should do any modifications?

deepakchethan commented 5 years ago

This can be closed