search

lunixbochs / usercorn

dynamic binary analysis via platform emulation
MIT License
890 stars 98 forks source link

aarch64 binaries (ARM64, ARMv8) #172

Open MagaTailor opened 8 years ago

MagaTailor commented 8 years ago

So I found this aarch64 busybox binary that's identified as:

busybox_static: ELF 64-bit LSB shared object, ARM aarch64, version 1 (SYSV), dynamically linked (uses shared libs), stripped

Usercorn says:

panic: Unsupported machine: EM_ALPHA_STD+142

Is it an android binary or sth? Should aarch64 linux binaries work?

lunixbochs commented 8 years ago

I haven't tested aarch64 anything yet. Don't even know the syscall ABI offhand. We do have an arm64 arch backend that has no OS support yet, and it looks like the ELF loader doesn't know about aarch64 either.

MagaTailor commented 8 years ago

Right, I'm aware there's just unicorn support for aarch64 at the moment, but was wondering if the transition to aarch64 would reuse much of the x86_64 Linux stuff, similar to x86 -> ARM previously.

lunixbochs commented 8 years ago

1972574e6877f3bc8b3dad747fcbc472dc48693a

Still need to wire up the kernel.

lunixbochs commented 8 years ago

Okay I wired the kernel and the android static binaries aren't static at all. They rely on the android arm64 linker /system/bin/linker64. I added a -nointerp flag and that didn't help.

MagaTailor commented 8 years ago

I'll have to find a Linux aarch64 busybox binary then. Here it is:

https://packages.debian.org/sid/arm64/busybox-static/download

MagaTailor commented 8 years ago 
$ ./usercorn -trace busybox-armv8 

[...]
+ block @0x50e410
    0x50e410:         ff4307d1 sub sp, sp, #0x1d0
    0x50e414:         fd7bbda9 stp x29, x30, [sp, #-0x30]!
    0x50e418:         fd030091 mov x29, sp
    0x50e41c:         a0e30191 add x0, x29, #0x78
    0x50e420:         f51300f9 str x21, [sp, #0x20]
    0x50e424:         f35301a9 stp x19, x20, [sp, #0x10]
    0x50e428:         76cffb97 bl #0x402200

    + block @0x402200
      0x402200:         081480d2 movz x8, #0xa0
      0x402204:         010000d4 svc #0          x8 0x00000000000000a0 

panic: unhandled ARM interrupt: 2

goroutine 17 [running, locked to thread]:
github.com/lunixbochs/usercorn/go/arch/arm64.LinuxInterrupt(0xaaa936e8, 0x10661960, 0x2)
        /home/odroid/go/src/github.com/lunixbochs/usercorn/go/arch/arm64/linux.go:40 +0x104
github.com/lunixbochs/usercorn/go.(*Usercorn).addHooks.func5(0xaaa93628, 0x106efbf0, 0x2)
        /home/odroid/go/src/github.com/lunixbochs/usercorn/go/usercorn.go:528 +0x84
github.com/unicorn-engine/unicorn/bindings/go/unicorn.hookInterrupt(0x4971f0, 0x2, 0x10890ea0)
        /home/odroid/go/src/github.com/unicorn-engine/unicorn/bindings/go/unicorn/hook.go:45 +0x80

goroutine 1 [syscall, locked to thread]:
github.com/unicorn-engine/unicorn/bindings/go/unicorn._Cfunc_uc_emu_start(0x4971e0, 0x40f920, 0x0, 0xffffffff, 0xffffffff, 0x0, 0x0, 0x0, 0x0)
        ??:0 +0x34
github.com/unicorn-engine/unicorn/bindings/go/unicorn.(*uc).StartWithOptions(0x106efbf0, 0x40f920, 0x0, 0xffffffff, 0xffffffff, 0x1064b9ac, 0x0, 0x0)
        /home/odroid/go/src/github.com/unicorn-engine/unicorn/bindings/go/unicorn/unicorn.go:91 +0x70
github.com/unicorn-engine/unicorn/bindings/go/unicorn.(*uc).Start(0x106efbf0, 0x40f920, 0x0, 0xffffffff, 0xffffffff, 0x0, 0x0)
        /home/odroid/go/src/github.com/unicorn-engine/unicorn/bindings/go/unicorn/unicorn.go:96 +0x78
github.com/lunixbochs/usercorn/go.(*Usercorn).Start(0x10661960, 0x40f920, 0x0, 0xffffffff, 0xffffffff, 0x0, 0x0)
        /home/odroid/go/src/github.com/lunixbochs/usercorn/go/usercorn.go:270 +0x70
github.com/lunixbochs/usercorn/go.(*Usercorn).Run(0x10661960, 0x1066e110, 0x1, 0x2, 0x106f6000, 0x54, 0x80, 0x0, 0x0)
        /home/odroid/go/src/github.com/lunixbochs/usercorn/go/usercorn.go:227 +0x11b4
main.main()
        /home/odroid/usercorn-unstable/go/usercorn/cli.go:172 +0x16a0
lunixbochs commented 8 years ago

syscall is at least invoked in unstable now

panic: Kernel not found for syscall 'sched_get_priority_min'

ghostrace doesn't have arm64 syscalls so this is probably wrong :) I need to find something like asm/unistd*64.h from an arm64 linux machine.

MagaTailor commented 8 years ago

Yup, funny armv5-7 never needed that syscall. (sched_get_priority_min)

lunixbochs commented 8 years ago

The syscall seems bogus. My guess is the number table is wrong (I reused the 32-bit arm table). We need to harvest a 64-bit arm table.

MagaTailor commented 8 years ago

Ok, I've got an arm64 unistd32.h here, straight from 4.4.6 Linux source - how do I regenerate the table?

lunixbochs commented 8 years ago

I need unistd64 with all the NR_sys_write defines, just pastebin it and I'll include in ghostrace

MagaTailor commented 8 years ago

Have a look here - they don't seem to follow that naming convention.

lunixbochs commented 8 years ago

asm/unistd.h should be right then

MagaTailor commented 8 years ago

http://pastebin.com/nGYH487Y

Wasn't the file a little short for the purpose?

MagaTailor commented 8 years ago

I've converted my Cortex-A53 Android TV to Linux (remember that cool deal?) - what's the state of usercorn on aarch64?

lunixbochs commented 8 years ago

Emulating aarch64 bins, or running on an aarch64 host? I don't have any good hosts to test on, and I don't remember how well guest bins run. It's likely missing the kernel bridge.

MagaTailor commented 8 years ago

This issue was about bins, so let's just settle down for host :) No problems there?

lunixbochs commented 8 years ago

Oh yeah, still need to figure out the syscall number mapping.

MagaTailor commented 8 years ago

The new platform can run 32-bit ARM binaries natively so it seems the latter can be achieved by simply running a current ARMv7 build. 

odroid@mxqproplus:~/usercorn-kit$  LD_LIBRARY_PATH=~/usercorn-kit/ ./usercorn busybox-x86_64 cat /proc/swaps

Filename                                Type            Size    Used    Priority
/dev/zram0                              partition       911924  0       5
/dev/zram1                              partition       911924  0       5
/dev/zram2                              partition       911924  0       5
/dev/zram3                              partition       911924  0       5
MagaTailor commented 7 years ago

Which parts of the usercorn framework are still missing for native aarch64 support to materialise?

kkszysiu commented 6 years ago

Hey guys,

I'm also interested in aarch64 support.

lunixbochs commented 6 years ago

Is the primary interest for Linux or Android?

kkszysiu commented 6 years ago

@lunixbochs in my case it's Linux :)

kkszysiu commented 6 years ago

@lunixbochs do you have the source code of examples from ./bins/ directory? I can compile it on Aarch64 and make a PR for easier debugging.

lunixbochs commented 6 years ago

That's from my lib43 project which doesn't have aarch64 support yet (though it would be easy to add).

For basic debugging Busybox is a good start.

kkszysiu commented 6 years ago

So far I've only compiled simple C hello world:

#include <stdio.h>

int main(int argc, char *argv[])
{
  printf("Hello AArch64!\n");
  return 0;
}
file ./bins/x86_64.linux.elf
./bins/x86_64.linux.elf: ELF 64-bit LSB executable, x86-64, version 1 (SYSV), statically linked, BuildID[sha1]=fcc18bd0b6522f57e9f523c82a1cd19addba3722, not stripped

And this is what usercorn is returning while executing that file:

./usercorn ./bins/arm64.linux.elf

+++ caught panic +++
Kernel not found for syscall 'sched_get_priority_min'

[memory map]
[registers]
[callstack]
  0x0
%!(EXTRA string=0x0)panic: Kernel not found for syscall 'sched_get_priority_min' [recovered]
    panic: Kernel not found for syscall 'sched_get_priority_min'

goroutine 1 [running]:
github.com/lunixbochs/usercorn/go.(*Usercorn).Run.func1(0xc420142200)
    /Users/kkszysiu/Projects/Stuff/usercorn/.gopath/src/github.com/lunixbochs/usercorn/go/usercorn.go:399 +0x4f2
panic(0x43efe80, 0xc4201e2ea0)
    /Users/kkszysiu/Projects/Stuff/usercorn/deps/go-x86_64-Darwin/src/runtime/panic.go:491 +0x283
github.com/lunixbochs/usercorn/go.(*Usercorn).Syscall(0xc420142200, 0xa0, 0x4464998, 0x16, 0xc420144570, 0xc420144570, 0x12, 0x0)
    /Users/kkszysiu/Projects/Stuff/usercorn/.gopath/src/github.com/lunixbochs/usercorn/go/usercorn.go:800 +0x7d3
github.com/lunixbochs/usercorn/go/arch/arm64.LinuxSyscall(0x46ce160, 0xc420142200, 0xa0)
    /Users/kkszysiu/Projects/Stuff/usercorn/.gopath/src/github.com/lunixbochs/usercorn/go/arch/arm64/linux.go:38 +0xd8
github.com/lunixbochs/usercorn/go/arch/arm64.LinuxInterrupt(0x46ce160, 0xc420142200, 0xc400000002)
    /Users/kkszysiu/Projects/Stuff/usercorn/.gopath/src/github.com/lunixbochs/usercorn/go/arch/arm64/linux.go:45 +0x65
github.com/lunixbochs/usercorn/go.(*Usercorn).addHooks.func2(0x46c9620, 0xc420048b50, 0x2)
    /Users/kkszysiu/Projects/Stuff/usercorn/.gopath/src/github.com/lunixbochs/usercorn/go/usercorn.go:597 +0x49
github.com/lunixbochs/usercorn/go/cpu/unicorn.(*UnicornCpu).HookAdd.func3(0x46ca340, 0xc42000cda0, 0xc400000002)
    /Users/kkszysiu/Projects/Stuff/usercorn/.gopath/src/github.com/lunixbochs/usercorn/go/cpu/unicorn/unicorn.go:53 +0x45
github.com/unicorn-engine/unicorn/bindings/go/unicorn.hookInterrupt(0x7000000, 0x2, 0x1)
    /Users/kkszysiu/Projects/Go/src/github.com/unicorn-engine/unicorn/bindings/go/unicorn/hook.go:80 +0x71
github.com/unicorn-engine/unicorn/bindings/go/unicorn._cgoexpwrap_5df61e954c57_hookInterrupt(0x7000000, 0x10e00000002, 0x1)
    github.com/unicorn-engine/unicorn/bindings/go/unicorn/_obj/_cgo_gotypes.go:637 +0x3d
github.com/unicorn-engine/unicorn/bindings/go/unicorn._Cfunc_uc_emu_start(0x7000000, 0x400958, 0xffffffffffffffff, 0x0, 0x0, 0xc400000000)
    github.com/unicorn-engine/unicorn/bindings/go/unicorn/_obj/_cgo_gotypes.go:236 +0x50
github.com/unicorn-engine/unicorn/bindings/go/unicorn.(*uc).StartWithOptions.func1(0x7000000, 0x400958, 0xffffffffffffffff, 0x0, 0x0, 0x1)
    /Users/kkszysiu/Projects/Go/src/github.com/unicorn-engine/unicorn/bindings/go/unicorn/unicorn.go:110 +0x8c
github.com/unicorn-engine/unicorn/bindings/go/unicorn.(*uc).StartWithOptions(0xc42000cda0, 0x400958, 0xffffffffffffffff, 0xc4200d3950, 0xc42001bbb0, 0xc42001bbb8)
    /Users/kkszysiu/Projects/Go/src/github.com/unicorn-engine/unicorn/bindings/go/unicorn/unicorn.go:110 +0x5c
github.com/unicorn-engine/unicorn/bindings/go/unicorn.(*uc).Start(0xc42000cda0, 0x400958, 0xffffffffffffffff, 0xc4200d39d0, 0x43482ab)
    /Users/kkszysiu/Projects/Go/src/github.com/unicorn-engine/unicorn/bindings/go/unicorn/unicorn.go:115 +0x5b
github.com/lunixbochs/usercorn/go/cpu/unicorn.(*UnicornCpu).Start(0xc420048b50, 0x400958, 0xffffffffffffffff, 0x0, 0x400958)
    <autogenerated>:1 +0x53
github.com/lunixbochs/usercorn/go.(*Usercorn).Start(0xc420142200, 0x400958, 0xffffffffffffffff, 0x0, 0x0)
    /Users/kkszysiu/Projects/Stuff/usercorn/.gopath/src/github.com/lunixbochs/usercorn/go/usercorn.go:511 +0x54
github.com/lunixbochs/usercorn/go.(*Usercorn).Run(0xc420142200, 0x0, 0x0)
    /Users/kkszysiu/Projects/Stuff/usercorn/.gopath/src/github.com/lunixbochs/usercorn/go/usercorn.go:455 +0x316
github.com/lunixbochs/usercorn/go/cmd.(*UsercornCmd).Run(0xc42008a2d0, 0xc42000c060, 0x2, 0x2, 0xc420010580, 0x2c, 0x2c, 0x0)
    /Users/kkszysiu/Projects/Stuff/usercorn/.gopath/src/github.com/lunixbochs/usercorn/go/cmd/cmd.go:426 +0x1dab
main.main()
    /Users/kkszysiu/Projects/Stuff/usercorn/go/cmd/usercorn/main.go:10 +0x83
lunixbochs commented 6 years ago

I think the arm64 linux syscall number tables need to be tracked down

lunixbochs commented 6 years ago

arm64 busybox is working in latest unstable (make sure to run make get to update ghostrace and get the syscall tables)

MagaTailor commented 6 years ago

I've built usercorn on ARMv7 and it doesn't work for me. Are there any special requirements re: unicorn revision or is it just a 32-bit issue?

$ ./usercorn busybox-armv8 

+++ caught panic +++
unhandled ARM interrupt: 1

[memory map]
[registers]
[callstack]
  0x0
panic: unhandled ARM interrupt: 1 [recovered]
        panic: unhandled ARM interrupt: 1

goroutine 1 [running]:
github.com/lunixbochs/usercorn/go.(*Usercorn).Run.func1(0x109b8640)
        ../github.com/lunixbochs/usercorn/go/usercorn.go:399 +0xc4
panic(0x366980, 0x10bba560)
        ../go1.8/src/runtime/panic.go:489 +0x288
github.com/lunixbochs/usercorn/go/arch/arm64.LinuxInterrupt(0x5c1b20, 0x109b8640, 0x1)
        ../github.com/lunixbochs/usercorn/go/arch/arm64/linux.go:48 +0xec
github.com/lunixbochs/usercorn/go.(*Usercorn).addHooks.func2(0x5bf0b8, 0x109706a0, 0x1)
        ../github.com/lunixbochs/usercorn/go/usercorn.go:597 +0x3c
github.com/lunixbochs/usercorn/go/cpu/unicorn.(*UnicornCpu).HookAdd.func3(0x5bf8f0, 0x109f0e80, 0x1)
        ../github.com/lunixbochs/usercorn/go/cpu/unicorn/unicorn.go:53 +0x38
github.com/unicorn-engine/unicorn/bindings/go/unicorn.hookInterrupt(0x5ed780, 0x1, 0x1)
        ../github.com/unicorn-engine/unicorn/bindings/go/unicorn/hook.go:80 +0x5c
github.com/unicorn-engine/unicorn/bindings/go/unicorn._cgoexpwrap_5df61e954c57_hookInterrupt(0x5ed780, 0x1, 0x1)
        github.com/unicorn-engine/unicorn/bindings/go/unicorn/_obj/_cgo_gotypes.go:636 +0x2c
github.com/unicorn-engine/unicorn/bindings/go/unicorn._Cfunc_uc_emu_start(0x5ed780, 0x40f920, 0x0, 0xffffffff, 0xffffffff, 0x0, 0x0, 0x0, 0x0)
        github.com/unicorn-engine/unicorn/bindings/go/unicorn/_obj/_cgo_gotypes.go:235 +0x38
github.com/unicorn-engine/unicorn/bindings/go/unicorn.(*uc).StartWithOptions.func1(0x5ed780, 0x40f920, 0x0, 0xffffffff, 0xffffffff, 0x0, 0x0, 0x0, 0x134d8c)
        ../github.com/unicorn-engine/unicorn/bindings/go/unicorn/unicorn.go:110 +0x7c
github.com/unicorn-engine/unicorn/bindings/go/unicorn.(*uc).StartWithOptions(0x109f0e80, 0x40f920, 0x0, 0xffffffff, 0xffffffff, 0x1094bca0, 0x5ed780, 0x104)
        ../github.com/unicorn-engine/unicorn/bindings/go/unicorn/unicorn.go:110 +0x5c
github.com/unicorn-engine/unicorn/bindings/go/unicorn.(*uc).Start(0x109f0e80, 0x40f920, 0x0, 0xffffffff, 0xffffffff, 0x0, 0x2ff718)
        ../github.com/unicorn-engine/unicorn/bindings/go/unicorn/unicorn.go:115 +0x60
github.com/lunixbochs/usercorn/go/cpu/unicorn.(*UnicornCpu).Start(0x109706a0, 0x40f920, 0x0, 0xffffffff, 0xffffffff, 0x40f920, 0x0)
        <autogenerated>:18 +0x6c
github.com/lunixbochs/usercorn/go.(*Usercorn).Start(0x109b8640, 0x40f920, 0x0, 0xffffffff, 0xffffffff, 0x0, 0x4c)
        ../github.com/lunixbochs/usercorn/go/usercorn.go:511 +0x54
github.com/lunixbochs/usercorn/go.(*Usercorn).Run(0x109b8640, 0x0, 0x0)
        ../github.com/lunixbochs/usercorn/go/usercorn.go:455 +0x278
github.com/lunixbochs/usercorn/go/cmd.(*UsercornCmd).Run(0x1096e9f0, 0x10982040, 0x2, 0x2, 0x1095ec80, 0x4c, 0x4c, 0x0)
        ../github.com/lunixbochs/usercorn/go/cmd/cmd.go:426 +0x1824
main.main()
        ../usercorn-unstable/go/cmd/usercorn/main.go:10 +0x68

All other architectures work fine.

lunixbochs commented 6 years ago

are you on the latest unstable branch? any output with the -trace flag?

MagaTailor commented 6 years ago

Yes, latest unstable, see the attached trace log. armv8-log.txt

lunixbochs commented 6 years ago

That looks like a NEON exception. I probably need to initialize the FPU

lunixbochs commented 6 years ago

Can you provide a link to your test binary?

MagaTailor commented 6 years ago

Here you are: busybox.zip