Not working on Android 10

summershrimp commented 5 years ago

Workaround at #3543 not work anymore. This time seems some bug DR dealing with bionic libc. After diffing bionic between P and Q beta, It seems that TLS structure is changed. I'm not sure this is the problem. Firstly I think it is the issue of the newer kernel Linux localhost 4.14.69 #1 SMP PREEMPT Wed Jun 5 23:46:39 CST 2019 aarch64 Android But I tried DR with Linux 4.19 and glibc, it works fine. So it might be the bionic libc.

The target behavior: Static link glibc no-pie running under Android Q: OK Static link glibc pie running under Android Q: OK Static link bionic no-pie running under Android Q: OK Static link bionic pie running under Android Q: Not compiling Dynamic link bionic no-pie running under Android Q: OK (Not sure why DR bypass linker PIE checking) Dynamic link bionic pie running under Android Q: Stuck at some point

And I strace the DR and target with strace -f ./bin64/drrun -c ./samples/bin64/libinscount.so -- ../hello_pie, DR clone it self and creates a lot of subprocess as the same args as it self.

Debug mode would trigger ASSERT

$ ./bin64/drrun -debug  -c ./samples/bin64/libinscount.so -- ../hello_pie
<Starting application /data/data/com.termux/files/home/hello_pie (30046)>
<Initial options = -no_dynamic_options -client_lib '/data/data/com.termux/files/home/DynamoRIO-Linux-7.91.18058-0/samples/bin64/libinscount.so;0;' -code_api -stack_size 56K -signal_stack_size 32K -max_elide_jmp 0 -max_elide_call 0 -early_inject -emulate_brk -no_inline_ignored_syscalls -native_exec_default_list '' -no_native_exec_managed_code -no_indcall2direct >
<Paste into GDB to debug DynamoRIO clients:
set confirm off
add-symbol-file '/data/data/com.termux/files/home/DynamoRIO-Linux-7.91.18058-0/samples/bin64/libinscount.so' 0x0000007e55ea1670
add-symbol-file '/data/data/com.termux/files/home/DynamoRIO-Linux-7.91.18058-0/lib64/debug/libdynamorio.so' 0x0000007e99ebf1e0
add-symbol-file '/data/data/com.termux/files/home/DynamoRIO-Linux-7.91.18058-0/ext/lib64/debug/libdrmgr.so' 0x0000007e55f162f0
add-symbol-file '/system/lib64/libm.so' 0x0000007e9a72a000
add-symbol-file '/system/lib64/libc.so' 0x0000007e99be3000
add-symbol-file '/system/lib64/ld-android.so' 0x0000007e9a715000
add-symbol-file '/system/lib64/libdl.so' 0x0000007e9a710000
>
Client inscount is running
<get_memory_info mismatch! (can happen if os combines entries in /proc/pid/maps)
        os says: 0x0000007e9a6cd000-0x0000007e9a6d7000 prot=0x00000003
        cache says: 0x0000007e9a6d4000-0x0000007e9a6d5000 prot=0x00000003
>
<(1+x) Handling our fault in a TRY at 0x0000007e99f788a8>
<Application /data/data/com.termux/files/home/hello_pie (30046).  Internal Error: DynamoRIO debug check failure: /home/xm1994/Projects/dynamorio/core/vmareas.c:8175 is_readable_without_exception_try(pc, 1)
(Error occurred @4220 frags)
version 7.91.18058, custom build
-no_dynamic_options -client_lib '/data/data/com.termux/files/home/DynamoRIO-Linux-7.91.18058-0/samples/bin64/libinscount.so;0;' -code_api -stack_size 56K -signal_stack_size 32K -max_elide_jmp 0 -max_elide_call 0 -early_inject -emulate_brk -no_inline_ignored_syscalls -native_exec_default_list '' -no_native_exec_managed_co
0x0000007dd5f127b0 0x0000007e99f71b34
0x0000007dd5f12950 0x0000007e9a039590
0x0000007dd5f12b90 0x0000007e9a17181c
0x0000007dd5f12cf0 0x0000007e9a17da94
0x0000007dd5f12d30 0x0000007e9a17ded4
0x0000007dd5f12d80 0x0000007e99f56f34
0x0000007dd5f12f40 0x0000007e9a60db6c
0x0000007fff64fa10 0x0000007e9a60dd88
0x0000007fff64fa90 0x0000007e9a60dc94
0x0000007fff64fb10 0x0000007e9a60dc94
0x0000007fff64fb90 0x0000007e9a609d68
0x0000007fff650d30 0x0000007e9a608f38
0x0000007fff651020 0x0000007e9a6101d8
/data/data/com.termux/files/home/DynamoRIO-Linux-7.91.18058-0/samples/bin64/libinscount.so=0x0000007e55e9a000
/system/lib64/libm.so=0x0000007e9a718000
/system/lib64/libc.so=0x0000007e99ba2000
/system/lib64/libdl.so=0x0000007e9a70f000
/system/lib64/ld-android.so=0x0000007e9a714000
/data/data/com.termux/files/home/DynamoRIO-Linux-7.91.18058-0/ext/lib64/debug/libdrmgr.so=0x0000007e55f13000>

AssadHashmi commented 5 years ago

Your analysis and the relevant DR code, suggests the newer bionic file size and/or its sections' sizes probably result in loading/mapping error(s). Can you run with -debug -loglevel 3 and attach the resulting log files please?

summershrimp commented 5 years ago

-rw-------    1 u0_a160  u0_a160   462.1K Jun 17 09:54 hello_pie.0.12639.html
-rw-------    1 u0_a160  u0_a160    44.9M Jun 17 09:54 log.0.12639.html

Hmm.. maybe too large.

derekbruening commented 5 years ago

Does it work with no client?

summershrimp commented 5 years ago

No, it doesn't work with no client.

$ ./bin64/drrun -- ls
Stuck...

$ strace -f ./bin64/drrun -- ls
...........
[pid 19956] gettid()                    = 19956         
[pid 19956] gettid()                    = 19956
[pid 19956] gettid()                    = 19956
[pid 19956] gettid()                    = 19956
[pid 19956] gettid()                    = 19956
[pid 19956] gettid()                    = 19956
[pid 19956] gettid()                    = 19956
[pid 19956] gettid()                    = 19956
[pid 19956] gettid()                    = 19956
[pid 19956] gettid()                    = 19956
[pid 19956] gettid()                    = 19956
[pid 19956] gettid()                    = 19956
[pid 19956] gettid()                    = 19956
[pid 19956] gettid()                    = 19956
[pid 19956] gettid()                    = 19956
[pid 19956] gettid()                    = 19956
[pid 19956] gettid()                    = 19956
[pid 19956] gettid()                    = 19956
[pid 19956] gettid()                    = 19956
[pid 19956] gettid()                    = 19956
[pid 19956] gettid()                    = 19956
[pid 19956] gettid()                    = 19956
[pid 19956] gettid()                    = 19956
[pid 19956] gettid()                    = 19956
[pid 19956] gettid()                    = 19956
[pid 19956] gettid()                    = 19956
[pid 19956] gettid()                    = 19956
[pid 19956] gettid()                    = 19956
[pid 19956] gettid()                    = 19956
[pid 19956] gettid()                    = 19956
[pid 19956] gettid()                    = 19956
[pid 19956] gettid()                    = 19956
[pid 19956] gettid()                    = 19956
[pid 19956] gettid()                    = 19956
[pid 19956] gettid()                    = 19956
[pid 19956] gettid()                    = 19956
[pid 19956] gettid()                    = 19956
[pid 19956] gettid()                    = 19956
[pid 19956] gettid()                    = 19956
[pid 19956] gettid()                    = 19956
^C[pid 19956] --- SIGINT {si_signo=SIGINT, si_code=SI_KERNEL} ---
strace: Process 19732 detached
strace: Process 19794 detached
strace: Process 19798 detached
strace: Process 19831 detached
strace: Process 19854 detached
strace: Process 19896 detached
strace: Process 19899 detached
strace: Process 19955 detached
strace: Process 19956 detached

$ ./bin64/drrun -debug -- ls
<Starting application /data/data/com.termux/files/usr/bin/busybox (23390)>
<Initial options = -no_dynamic_options -code_api -stack_size 56K -signal_stack_size 32K -max_elide_jmp 0 -max_elide_call 0 -early_inject -emulate_brk -no_inline_ignored_syscalls -native_exec_default_list '' -no_native_exec_managed_code -no_indcall2direct >
<Paste into GDB to debug DynamoRIO clients:
set confirm off
add-symbol-file '/data/data/com.termux/files/home/DynamoRIO-Linux-7.91.18058-0/lib64/debug/libdynamorio.so' 0x00000073061d81e0
>
<get_memory_info mismatch! (can happen if os combines entries in /proc/pid/maps)
        os says: 0x00000073069e6000-0x00000073069f0000 prot=0x00000003
        cache says: 0x00000073069ed000-0x00000073069ee000 prot=0x00000003
>
<(1+x) Handling our fault in a TRY at 0x00000073062918a8>
<Application /data/data/com.termux/files/usr/bin/busybox (23390).  Internal Error: DynamoRIO debug check failure: /home/xm1994/Projects/dynamorio/core/vmareas.c:8175 is_readable_without_exception_try(pc, 1)
(Error occurred @4307 frags)
version 7.91.18058, custom build
-no_dynamic_options -code_api -stack_size 56K -signal_stack_size 32K -max_elide_jmp 0 -max_elide_call 0 -early_inject -emulate_brk -no_inline_ignored_syscalls -native_exec_default_list '' -no_native_exec_managed_code -no_indcall2direct
0x00000072421ac5f0 0x000000730628ab34
0x00000072421ac790 0x0000007306352590
0x00000072421ac9d0 0x000000730648a81c
0x00000072421acb30 0x000000730648ef44
0x00000072421acb70 0x0000007306496f40
0x00000072421acd80 0x000000730626ff34
0x00000072421acf40 0x0000007306926b6c
0x0000007ffbfff600 0x0000007306926d88
0x0000007ffbfff680 0x0000007306926c94
0x0000007ffbfff700 0x0000007306926c94
0x0000007ffbfff780 0x0000007306922d68
0x0000007ffc000920 0x0000007306921f38
0x0000007ffc000c10 0x00000073069291d8>

summershrimp commented 1 year ago

Update: this may caused by Android 10 XoM enabled, but it disabled after android 11.

DynamoRIO / dynamorio

Not working on Android 10 #3683