app_process32 crashes on Android O

ghost commented 5 years ago

I'm trying to run app_process32 with drrun on my Pixel 2 (Android 8.1.0) but it crashes with SIGSEGV instantly every time (even without any argument)

walleye:/data/ssh/root # /system/xbin/dynamorio/bin32/drrun -- app_process32

<Application /system/bin/app_process32 (28241).  DynamoRIO internal crash at PC 0xf6326f30.  Please report this at http://dynamorio.org/issues/.  Program aborted.
Received SIGSEGV at pc 0xf6326f30 in thread 28241
Base: 0xf628b000
Registers:  r0 =0x00000000 r1 =0x00000000 r2 =0x0000f7ff r3 =0xf7ffbffe
        r4 =0x00000000 r5 =0x4bc46a84 r6 =0x00000000 r7 =0x00000000
        r8 =0x4bc40c88 r9 =0x4bc40c90 r10=0x00000000 r11=0x00000001
        r12=0x00004c22 r13=0x4bc40c80 r14=0xf6326e25 r15=0xf6326f30
        eflags=0x200b1830
version 7.1.0, build 1
-no_dynamic_options -code_api -stack_size 56K -signal_stack_size 32K -max_elide_jmp 0 -max_elide_call 0 -early_inject -emulate_brk -no_inline_ignored_syscalls -native_exec_default_list '' -no_native_exec_managed_code -no_indcall2direct >

I'm using the release version DynamoRIO-ARM-Android-EABI-7.1.0-1.tar.gz.

According to addr2line it crashes at /dynamorio_package/core/arch/arm/encode.c:3084

    if (di.isa_mode == DR_ISA_ARM_THUMB) {
        if (di.instr_word >> 16 != 0) {
            *((ushort *)copy_pc) = (ushort)(di.instr_word >> 16);  // <- HERE
            copy_pc += THUMB_SHORT_INSTR_SIZE;
        }
        *((ushort *)copy_pc) = (ushort)di.instr_word;
        copy_pc += THUMB_SHORT_INSTR_SIZE;
    } else {
        *((uint *)copy_pc) = di.instr_word;
        copy_pc += ARM_INSTR_SIZE;
    }

btw I tried some simple commands like ls date wget ping etc. they all works perfectly fine

hgreving2304 commented 5 years ago

Maybe the ARM people can help if you post more about the instruction it is trying to encode. Can you run in gdb and post how the instr_t looks like, logs etc.?

On Sun, Apr 14, 2019 at 5:54 AM cra10 notifications@github.com wrote:

I'm trying to run app_process32 with drrun on my Pixel 2 (Android 8.1.0) but it crashes with SIGSEGV every single time (even without any argument at all)

walleye:/data/ssh/root # /system/xbin/dynamorio/bin32/drrun -- app_process32

<Application /system/bin/app_process32 (28241). DynamoRIO internal crash at PC 0xf6326f30. Please report this at http://dynamorio.org/issues/. Program aborted. Received SIGSEGV at pc 0xf6326f30 in thread 28241 Base: 0xf628b000 Registers: r0 =0x00000000 r1 =0x00000000 r2 =0x0000f7ff r3 =0xf7ffbffe r4 =0x00000000 r5 =0x4bc46a84 r6 =0x00000000 r7 =0x00000000 r8 =0x4bc40c88 r9 =0x4bc40c90 r10=0x00000000 r11=0x00000001 r12=0x00004c22 r13=0x4bc40c80 r14=0xf6326e25 r15=0xf6326f30 eflags=0x200b1830 version 7.1.0, build 1 -no_dynamic_options -code_api -stack_size 56K -signal_stack_size 32K -max_elide_jmp 0 -max_elide_call 0 -early_inject -emulate_brk -no_inline_ignored_syscalls -native_exec_default_list '' -no_native_exec_managed_code -no_indcall2direct >

I'm using the release version DynamoRIO-ARM-Android-EABI-7.1.0-1.tar.gz.

According to addr2line it crashes at /dynamorio_package/core/arch/arm/encode.c:3084
if (di.isa_mode == DR_ISA_ARM_THUMB) {
    if (di.instr_word >> 16 != 0) {
        *((ushort *)copy_pc) = (ushort)(di.instr_word >> 16);  // <- HERE
        copy_pc += THUMB_SHORT_INSTR_SIZE;
    }
    *((ushort *)copy_pc) = (ushort)di.instr_word;
    copy_pc += THUMB_SHORT_INSTR_SIZE;
} else {
    *((uint *)copy_pc) = di.instr_word;
    copy_pc += ARM_INSTR_SIZE;
}
btw I tried some simple commands like ls date wget ping etc. they all works perfectly fine

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/DynamoRIO/dynamorio/issues/3540, or mute the thread https://github.com/notifications/unsubscribe-auth/ApX4dVgRzWdb-gRNA4BaRyDS9GJ_xmKSks5vgyUEgaJpZM4cumZT .

ghost commented 5 years ago

Updated my issue a bit. I think the problem is caused by copy_pc == 0, still trying to find out why

AssadHashmi commented 5 years ago

Try drrun with the debug and tracing options, e.g. /system/xbin/dynamorio/bin32/drrun -debug -loglevel 3 -- app_process32 This should enable asserts to warn of any unexpected state/condition as well as tracking down the actual instruction which is causing the segfault, which should appear in the log file.

As it's crashing in the ARM codec, it could an instruction which hasn't been sufficiently tested. If it's the strh.w in your screen dump, it could be previous instruction(s) which set r6 which is/are buggy. The logfile should give you some idea.

ghost commented 5 years ago

It seems if I enable -debug it will fail at an assert earlier instead of crashing with segfault

/system/xbin/dynamorio/bin32/drrun -debug -loglevel 3 -- app_process32

<log dir=/system/xbin/dynamorio/bin32/../logs/app_process32.20608.00000000>
<Starting application /system/bin/app_process32 (20608)>
<Initial options = -no_dynamic_options -loglevel 3 -code_api -stack_size 56K -signal_stack_size 32K -max_elide_jmp 0 -max_elide_call 0 -early_inject -emulate_brk -no_inline_ignored_syscalls -native_exec_default_list '' -no_native_exec_managed_code -no_indcall2direct >
<Paste into GDB to debug DynamoRIO clients:
set confirm off
add-symbol-file '/system/xbin/dynamorio/lib32/debug/libdynamorio.so' 0xf67234f8
>
<get_memory_info mismatch! (can happen if os combines entries in /proc/pid/maps)
        os says: 0xf66b1000-0xf6700000 prot=0x00000003
        cache says: 0xf66f9000-0xf66fa000 prot=0x00000003
>
<Application /system/bin/app_process32 (20608).  Internal Error: DynamoRIO debug check failure: /dynamorio_package/core/unix/memcache.c:417 found
(Error occurred @23989 frags)
version 7.1.0, build 1
-no_dynamic_options -loglevel 3 -code_api -stack_size 56K -signal_stack_size 32K -max_elide_jmp 0 -max_elide_call 0 -early_inject -emulate_brk -no_inline_ignored_syscalls -native_exec_default_list '' -no_native_exec_managed_code -no_indcall2direct
0xf678f595 0x78af04b0>
<rank order violation shared_cache_lock(mutex)@/dynamorio_package/core/fcache.c:1582 acquired after all_memory_areas(readwrite)@/dynamorio_package/core/unix/memcache.c:101 in tid:5080>

AssadHashmi commented 5 years ago

It seems if I enable -debug it will fail at an assert earlier instead of crashing with segfault

Yes, it looks like the get_memory_info assert has happened before the SIGSEGV in the codec. Can you try the -ignore_assert_list option to get past that? See http://dynamorio.org/docs/using.html#sec_options for usage.

What we want to get is the instruction trace in the log up to and including the faulting instruction to identify the sequence of instructions leading up to it, or any relevant asserts.

The other approach is GDB as suggested by @hgreving2304 above.

It's worth looking at https://github.com/DynamoRIO/dynamorio/wiki/Debugging and http://dynamorio.org/docs/using.html#sec_debugging in case you haven't come across them before.

The get_memory_info assert may not affect your app anyway, so not worth worrying about that yet.

ghost commented 5 years ago

this is the output of /system/xbin/dynamorio/bin32/drrun -debug -ignore_assert_list '*' -loglevel 3 -- app_process32

<log dir=/system/xbin/dynamorio/bin32/../logs/app_process32.22450.00000000>
<Starting application /system/bin/app_process32 (22450)>
<Initial options = -no_dynamic_options -loglevel 3 -code_api -stack_size 56K -signal_stack_size 32K -max_elide_jmp 0 -max_elide_call 0 -early_inject -emul                       ate_brk -no_inline_ignored_syscalls -native_exec_default_list '' -no_native_exec_managed_code -no_indcall2direct -ignore_assert_list '*' >
<Paste into GDB to debug DynamoRIO clients:
set confirm off
add-symbol-file '/system/xbin/dynamorio/lib32/debug/libdynamorio.so' 0xe9ba64f8
>
<get_memory_info mismatch! (can happen if os combines entries in /proc/pid/maps)
        os says: 0xe9b34000-0xe9b83000 prot=0x00000003
        cache says: 0xe9b7c000-0xe9b7d000 prot=0x00000003
>
<Ignoring assert /dynamorio_package/core/vmareas.c:1489 start < end || end == NULL>
<Ignoring assert /dynamorio_package/core/vmareas.c:1489 start < end || end == NULL>
<Ignoring assert /dynamorio_package/core/vmareas.c:1489 start < end || end == NULL>
<Ignoring assert /dynamorio_package/core/vmareas.c:1489 start < end || end == NULL>
<Ignoring assert /dynamorio_package/core/vmareas.c:1489 start < end || end == NULL>
<Ignoring assert /dynamorio_package/core/vmareas.c:924 start < end>
<Ignoring assert /dynamorio_package/core/unix/memcache.c:417 found>
<Application /system/bin/app_process32 (22450) DynamoRIO usage error : instr_decode: raw bits are invalid>
<Usage error: instr_decode: raw bits are invalid (/dynamorio_package/core/arch/instr_shared.c, line 1548)
version 7.1.0, build 1
-no_dynamic_options -loglevel 3 -code_api -stack_size 56K -signal_stack_size 32K -max_elide_jmp 0 -max_elide_call 0 -early_inject -emulate_brk -no_inline_ignored_syscalls -native_exec_default_list '' -no_native_exec_managed_code -no_indcall2direct -ignore_assert_list '*'
0xe9c12595 0x78af04b0>

some possibly useful entries from log file:

exit_branch_type=0x9 bb->exit_target=0xe60972e4
exit_branch_type=0x9 target=0xe60972e4 l->flags=0x1809
Exit cti 0x4e564ca4 is targeting 0x4e564ca8 + 0x0 => 0x4e564ca8
Fragment 34459, tag 0xe5fb46fc, flags 0x1400030, shared, size 40:

Entry into F34459(0xe5fb46fc).0x4e564c98 (T32)(shared)

Exit from F25834(0xe6097432).0x4e438070 (shared)    
 (target 0xe5fb4704 not in cache)
fragment_add_ibl_target tag 0xe5fb4704, branch 0, F0 

dispatch: target = 0xe5fb4704

interp: start_pc = 0xe5fb4704
  0xe5fb4704  68e9       ldr    +0x0c(%r5)[4byte] -> %r1
  0xe5fb4706  4408       add    %r0 %r1 -> %r0
  0xe5fb4708  7929       ldrb   +0x04(%r5)[1byte] -> %r1
  0xe5fb470a  0649       lsls   %r1 $0x00000019 -> %r1
  0xe5fb470c  d44e       b.mi   $0xe5fb47ac
end_pc = 0xe5fb470e

exit_branch_type=0x0 bb->exit_target=0xe5fb470e
exit_branch_type=0x11 target=0xe5fb47ac l->flags=0x11
exit_branch_type=0x0 target=0xe5fb470e l->flags=0x1001
Exit cti 0x4e564ccc is targeting 0x4e564cd4 + 0x0 => 0x4e564cd4
Exit cti 0x4e564cd0 is targeting 0x4e564ce8 + 0x0 => 0x4e564ce8
Fragment 34460, tag 0xe5fb4704, flags 0x9400030, shared, size 60:

Entry into F34460(0xe5fb4704).0x4e564cc4 (T32)(shared)

Exit from F34460(0xe5fb4704).0x4e564ccc (shared)    
 (target 0xe5fb47ac not in cache)

dispatch: target = 0xe5fb47ac

interp: start_pc = 0xe5fb47ac
  0xe5fb47ac  e8d0 12ff  <INVALID>
interp: invalid instr at 0xe5fb47ac
end_pc = 0xe5fb47b0

SYSLOG_ERROR: Application /system/bin/app_process32 (22450) DynamoRIO usage error : instr_decode: raw bits are invalid
SYSLOG_ERROR: Usage error: instr_decode: raw bits are invalid (/dynamorio_package/core/arch/instr_shared.c, line 1548)
version 7.1.0, build 1
-no_dynamic_options -loglevel 3 -code_api -stack_size 56K -signal_stack_size 32K -max_elide_jmp 0 -max_elide_call 0 -early_inject -emulate_brk -no_inline_ignored_syscalls -native_exec_default_list '' -no_native_exec_managed_code -no_indcall2direct -ignore_assert_list '*' 
0xe9c12595 0x78af04b0
DynamoRIO call stack:
Thread 22450 call stack:
    frame ptr 0xe9c12595 => parent 0x9a43b0e9, 0x78af04b0

file app_process32.0.22450.html & log.0.22450.html

https://s.put.re/r1eRSkxN.zip

derekbruening commented 5 years ago

So it's running along in Thumb mode and the app branches to a bogus instruction? 2 of my 3 decoders think it's invalid but they may not be as up to date as they could be:

$ disasm_tw e8d0 12ff
<stdin>:1:1: warning: invalid instruction encoding
0xd0 0xe8 0xff 0x12
^
<stdin>:1:6: warning: invalid instruction encoding
0xd0 0xe8 0xff 0x12
     ^
llvm-mc:   e8d0 asrs r7, r7, #11
capstone:  e8d0 12ff <INVALID: errcode 0>
bfd:       e8d0 12ff ldaexd r1, r2, [r0]

Is this something new: ARMv8.4 or sthg? @cra10: What do your tools say: objdump, or gdb, for that address? @AssadHashmi maybe you know more here: is this a valid instruction.

If it's invalid: did DR go on the wrong track earlier and this is not regular code? @cra10: which part of the binary is this? With debug info, does gdb think this is inside a function?