search

unicorn-engine / unicorn

Unicorn CPU emulator framework (ARM, AArch64, M68K, Mips, Sparc, PowerPC, RiscV, S390x, TriCore, X86)
http://www.unicorn-engine.org
GNU General Public License v2.0
7.34k stars 1.31k forks source link

Zig - Sample build #1815

Closed kassane closed 11 months ago

kassane commented 1 year ago

The sample riscv ( random choose) was chosen for reference and was used as basis for developing the wrapper/binding. In addition to a basic build-system that could trigger cmake and its necessary commands to build.

How to use

Note: It is recommended that you try to link the libunicorn-shared or build libunicorn with the same compiler (zig cc). Unfortunately there are conflicts with gcc/mingw.

Ref.: #1809

kassane commented 1 year ago

Hi @lupyuen,

I remember you made a pinephone guide using unicorn-rs. Do you have any suggestions for suitable zig support based on your experiment?

lupyuen commented 1 year ago

@kassane Sorry I haven't thought about how we might support Zig in Unicorn. (I used the standard Rust Bindings for Unicorn)

But I think Zig support would be really useful.

kassane commented 1 year ago

@wtdcode Test outupt (Zig CI test): https://github.com/kassane/unicorn/actions/runs/4596902849/jobs/8118942852

--edit

https://github.com/kassane/unicorn/actions/runs/4640759805

wtdcode commented 1 year ago

Sorry for late because I was trying to use zig to try your samples and stuck on some dumb questions. I will review it asap.

kassane commented 1 year ago

Sorry for late because I was trying to use zig to try your samples and stuck on some dumb questions. I will review it asap.

No worries! If you need help, I'm available.

kassane commented 1 year ago

Post commit https://github.com/unicorn-engine/unicorn/pull/1815/commits/765c84dd943901a2dca1b1750eef31d2c9a51f72 was retested on windows mingw and no segfault occurred. Currently zig toolchain/mingw doesn't provide winpthreads support. However, in this build the libunicorn already includes winpthread.

Before **Msys2 terminal** ```bash Matheus@DESKTOP-BUHRSIO CLANG64 ~/unicorn $ PATH=$PATH:$PWD/build zig build run info(unicorn): Emulate RISCV code: recover_from_illegal info(unicorn): >>> Allocating block at 0x4096 (0x4096), block size = 0x2 (0x4096) info(unicorn): >>> Tracing basic block at 0x4096, block size = 0x0 error(unicorn): Expected Illegal Instruction error, got: error.EXCEPTION info(unicorn): >>> Tracing basic block at 0x65536, block size = 0x8 info(unicorn): >>> Tracing instruction at 0x65536, instruction size = 0x4 info(unicorn): >>> Emulation done. Below is the CPU context info(unicorn): >>> A0 = 0x1 info(unicorn): >>> A1 = 0x30864 info(unicorn): ------------------ info(unicorn): Emulate RISCV code: split emulation info(unicorn): >>> Tracing basic block at 0x65536, block size = 0x4 info(unicorn): >>> Tracing basic block at 0x65536, block size = 0x4 info(unicorn): >>> A0 = 0x1 info(unicorn): >>> A1 = 0x30864 info(unicorn): >>> Tracing basic block at 0x65540, block size = 0x4 info(unicorn): >>> Tracing basic block at 0x65540, block size = 0x4 info(unicorn): >>> Emulation done. Below is the CPU context info(unicorn): >>> A0 = 0x1 info(unicorn): >>> A1 = 0x30864 Segmentation fault at address 0x18d4e268270 thread 11496 panic: start index 18181400 is larger than end index 17866752 C:\Users\Matheus\Downloads\zig\master\files\lib\std\coff.zig:1213:54: 0x7ff73974ef9b in getStrtab (sample_riscv.exe.obj) const size = mem.readIntLittle(u32, self.data[offset..][0..4]); ^ C:\Users\Matheus\Downloads\zig\master\files\lib\std\coff.zig:1235:42: 0x7ff73973c3c5 in getSectionName (sample_riscv.exe.obj) const strtab = self.getStrtab().?; ^ C:\Users\Matheus\Downloads\zig\master\files\lib\std\coff.zig:1244:48: 0x7ff739706837 in getSectionByName__anon_10896 (sample_riscv.exe.obj) if (mem.eql(u8, self.getSectionName(sect), name)) { ^ C:\Users\Matheus\Downloads\zig\master\files\lib\std\debug.zig:884:38: 0x7ff73972448d in readCoffDebugInfo (sample_riscv.exe.obj) if (coff_obj.getSectionByName(".debug_info")) |sec| { ^ C:\Users\Matheus\Downloads\zig\master\files\lib\std\debug.zig:1414:54: 0x7ff73970578e in lookupModuleWin32 (sample_riscv.exe.obj) obj_di.* = try readCoffDebugInfo(self.allocator, mapped_module); ^ C:\Users\Matheus\Downloads\zig\master\files\lib\std\debug.zig:1338:42: 0x7ff73970542f in getModuleForAddress (sample_riscv.exe.obj) return self.lookupModuleWin32(address); ^ C:\Users\Matheus\Downloads\zig\master\files\lib\std\debug.zig:767:50: 0x7ff739735c36 in printSourceAtAddress__anon_10126 (sample_riscv.exe.obj) const module = debug_info.getModuleForAddress(address) catch |err| switch (err) { ^ C:\Users\Matheus\Downloads\zig\master\files\lib\std\debug.zig:186:29: 0x7ff73975c3b4 in dumpStackTraceFromBase (sample_riscv.exe.obj) printSourceAtAddress(debug_info, stderr, ip, tty_config) catch return; ^ C:\Users\Matheus\Downloads\zig\master\files\lib\std\debug.zig:2069:45: 0x7ff73974b262 in handleSegfaultWindowsExtra__anon_14009 (sample_riscv.exe.obj) dumpStackTraceFromBase(regs.bp, regs.ip); ^ C:\Users\Matheus\Downloads\zig\master\files\lib\std\debug.zig:2046:73: 0x7ff739738840 in handleSegfaultWindows (sample_riscv.exe.obj) windows.EXCEPTION_ACCESS_VIOLATION => handleSegfaultWindowsExtra(info, 1, null), ^ ???:?:?: 0x7ffb9aac7b09 in ??? (???) ???:?:?: 0x7ffb9aa6e8c1 in ??? (???) ???:?:?: 0x7ffb9aaf2e9d in ??? (???) Panicked during a panic. Aborting. run sample_riscv: error: the following command exited with error code 3: C:\msys64\home\Matheus\unicorn\zig-out\bin\sample_riscv.exe ```
After **Msys2 terminal** ```bash Matheus@DESKTOP-BUHRSIO CLANG64 ~/unicorn $ PATH=$PATH:$PWD/build zig build run info(unicorn): Emulate RISCV code: recover_from_illegal info(unicorn): >>> Allocating block at 0x4096 (0x4096), block size = 0x2 (0x4096) info(unicorn): >>> Tracing basic block at 0x4096, block size = 0x0 error(unicorn): Expected Illegal Instruction error, got: error.EXCEPTION info(unicorn): >>> Tracing basic block at 0x65536, block size = 0x8 info(unicorn): >>> Tracing instruction at 0x65536, instruction size = 0x4 info(unicorn): >>> Emulation done. Below is the CPU context info(unicorn): >>> A0 = 0x1 info(unicorn): >>> A1 = 0x30864 info(unicorn): ------------------ info(unicorn): Emulate RISCV code: split emulation info(unicorn): >>> Tracing basic block at 0x65536, block size = 0x4 info(unicorn): >>> Tracing basic block at 0x65536, block size = 0x4 info(unicorn): >>> A0 = 0x1 info(unicorn): >>> A1 = 0x30864 info(unicorn): >>> Tracing basic block at 0x65540, block size = 0x4 info(unicorn): >>> Tracing basic block at 0x65540, block size = 0x4 info(unicorn): >>> Emulation done. Below is the CPU context info(unicorn): >>> A0 = 0x1 info(unicorn): >>> A1 = 0x30864 info(unicorn): ------------------ info(unicorn): Emulate RISCV code: return from func info(unicorn): ======== info(unicorn): >>> Tracing basic block at 0x65540, block size = 0x2 info(unicorn): >>> Tracing instruction at 0x65540, instruction size = 0x2 info(unicorn): >>> Tracing basic block at 0x65542, block size = 0x4 info(unicorn): Good, PC == RA info(unicorn): >>> Emulation done ```

sample_build.tar.gz

Edit Sadly still haven't finalized the readaptation from build.zig ( like standalone zig) to QEMU, because it needs to have the configs.h (host and target) predefined already in some build steps. The winpthread issue would probably get fixed with that: https://github.com/kassane/winpthreads-zigbuild

kassane commented 1 year ago

New output: (Build all samples)

Help

$> zig build -h

Usage: /home/kassane/zig/0.11.0-dev.2545+311d50f9d/files/zig build [steps] [options]

Steps:
  install (default)            Copy build artifacts to prefix path
  uninstall                    Remove build artifacts from prefix path
  sample_riscv_zig             Run the sample_riscv_zig sample
  sample_arm                   Run the sample_arm sample
  sample_arm64                 Run the sample_arm64 sample
  sample_ctl                   Run the sample_ctl sample
  sample_batch_reg             Run the sample_batch_reg sample
  sample_m68k                  Run the sample_m68k sample
  sample_riscv                 Run the sample_riscv sample
  sample_sparc                 Run the sample_sparc sample
  sample_s390x                 Run the sample_s390x sample
  shellcode                    Run the shellcode sample
  sample_tricore               Run the sample_tricore sample
  sample_x86                   Run the sample_x86 sample
  sample_x86_32_gdt_and_seg_regs Run the sample_x86_32_gdt_and_seg_regs sample
  cmake                        Run cmake build

Build

Build Summary: 29/29 steps succeeded
sample_x86_32_gdt_and_seg_regs success
└─ run sample_x86_32_gdt_and_seg_regs success 1ms MaxRSS:4M
   ├─ zig build-exe sample_x86_32_gdt_and_seg_regs Debug native success 355ms MaxRSS:353M
   └─ install success
      ├─ install sample_riscv_zig success
      │  └─ zig build-exe sample_riscv_zig Debug native success 2s MaxRSS:356M
      ├─ install sample_arm success
      │  └─ zig build-exe sample_arm Debug native success 344ms MaxRSS:352M
      ├─ install sample_arm64 success
      │  └─ zig build-exe sample_arm64 Debug native success 345ms MaxRSS:352M
      ├─ install sample_ctl success
      │  └─ zig build-exe sample_ctl Debug native success 368ms MaxRSS:352M
      ├─ install sample_batch_reg success
      │  └─ zig build-exe sample_batch_reg Debug native success 386ms MaxRSS:352M
      ├─ install sample_m68k success
      │  └─ zig build-exe sample_m68k Debug native success 415ms MaxRSS:353M
      ├─ install sample_riscv success
      │  └─ zig build-exe sample_riscv Debug native success 343ms MaxRSS:352M
      ├─ install sample_sparc success
      │  └─ zig build-exe sample_sparc Debug native success 527ms MaxRSS:353M
      ├─ install sample_s390x success
      │  └─ zig build-exe sample_s390x Debug native success 381ms MaxRSS:352M
      ├─ install shellcode success
      │  └─ zig build-exe shellcode Debug native success 1s MaxRSS:352M
      ├─ install sample_tricore success
      │  └─ zig build-exe sample_tricore Debug native success 241ms MaxRSS:351M
      ├─ install sample_x86 success
      │  └─ zig build-exe sample_x86 Debug native success 386ms MaxRSS:352M
      └─ install sample_x86_32_gdt_and_seg_regs success
         └─ zig build-exe sample_x86_32_gdt_and_seg_regs Debug native (reused)
kassane commented 1 year ago

cc: @wtdcode @aquynh

Following the theme proposed in the title of this contribution is enough. It only remains to improve the necessary later after the review.

The next step (another PR) will be to make zig a fully standalone build alternative (change build.zig).

Why? Would CMake not solve it?

zig is not well tolerated by cmake or any trivial build system because there is 'space' in the zig cc/zig c++ command causing certain conflicts. By using only the zig toolchain, it would allow, in theory, to unify the configuration between the library and the built examples (cross compilation).

wtdcode commented 1 year ago

cc: @wtdcode @aquynh

Following the theme proposed in the title of this contribution is enough. It only remains to improve the necessary later after the review.

The next step (another PR) will be to make zig a fully standalone build alternative (change build.zig).

Why? Would CMake not solve it?

zig is not well tolerated by cmake or any trivial build system because there is 'space' in the zig cc/zig c++ command causing certain conflicts. By using only the zig toolchain, it would allow, in theory, to unify the configuration between the library and the built examples (cross compilation).

Just curious about whether is there any other workaround for this? One straightforward workaround I think is something like ln -s "zig cc" "zigc".

kassane commented 1 year ago

Just curious about whether is there any other workaround for this? One straightforward workaround I think is something like ln -s "zig cc" "zigc".

Usually it has been tried to create a false clang (zigcc) in this way below:

Unix-like

#!/usr/bin/env bash
`which zig` cc $@

Windows - cmd

@echo off
zig cc %*

@mrexodia - zig-cmake https://github.com/mrexodia/zig-cross

mrexodia commented 1 year ago

The real issue with cmake is that tools like ranlib and friends cannot have arguments in the compiler path. As you can see in zig-cross this is worked around by pointing cmake to a script file that wraps it as mentioned above.

kassane commented 1 year ago

Latest change - CI: https://github.com/kassane/unicorn/actions/runs/5026896166

wtdcode commented 1 year ago

The only change I propose is to move the shell scripts to bindings/zig. Other looks fine to me. Sorry for the delay because I was sick for a few weeks.

kassane commented 1 year ago

zig-mingw build CI was disabled. It found a compiler failure (on zig cc w/ cmake), described here:

x86_64: https://github.com/kassane/unicorn/actions/runs/5032387218/jobs/9025992784#step:6:21

ERROR: Your compiler does not support the __thread specifier for 
       Thread-Local Storage (TLS). Please upgrade to a version that does.

aarch64-windows: https://github.com/kassane/unicorn/actions/runs/5032387218/jobs/9025992877#step:6:13

run cmake: error: unable to spawn cmake: InvalidExe   # x86_64 executable?
kassane commented 11 months ago

Zig 0.11.0 released: https://ziglang.org/download/0.11.0/release-notes.html

wtdcode commented 11 months ago

@kassane Is it the final version? I'm going to merge it if it's done.

kassane commented 11 months ago

@kassane Is it the final version? I'm going to merge it if it's done.

Yes! Merge - works in 0.11.0 stable

wtdcode commented 11 months ago

@kassane Is it the final version? I'm going to merge it if it's done.

Yes! Merge - works in 0.11.0 stable

Here you go. Thanks!