Build on PPC64LE with installed sysroot 2.28 fails

[MementoRC]

Zig Version

0.13.0

Steps to Reproduce and Observed Behavior

cmake build fails with:

$SRC_DIR/build-release/compiler_rt.c: In function 'truncl':
$SRC_DIR/build-release/compiler_rt.c:24291:2: warning: 'memcpy' reading 16 bytes from a region of size 8 [-Wstringop-overread]
24291 |  memcpy(&t1, &t0, sizeof(long double));
      |  ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
$SRC_DIR/build-release/compiler_rt.c:24287:10: note: source object 't0' of size 8
24287 |  zig_f64 t0;
      |          ^~

[17/19] Building C object CMakeFiles/zig2.dir/zig2.c.o

FAILED: CMakeFiles/zig2.dir/zig2.c.o 
$BUILD_PREFIX/bin/powerpc64le-conda-linux-gnu-cc -D_GNU_SOURCE -D__STDC_CONSTANT_MACROS -D__STDC_FORMAT_MACROS -D__STDC_LIMIT_MACROS -I$SRC_DIR/zig-source/stage1 -mcpu=power8 -mtune=power8 -ftree-vectorize -fPIC -fstack-protector-strong -fno-plt -O3 -pipe -isystem $PREFIX/include -fdebug-prefix-map=$SRC_DIR=/usr/local/src/conda/zig-0.13.0 -fdebug-prefix-map=$PREFIX=/usr/local/src/conda-prefix -O3 -DNDEBUG -std=c99 -O0 -fno-stack-protector -MD -MT CMakeFiles/zig2.dir/zig2.c.o -MF CMakeFiles/zig2.dir/zig2.c.o.d -o CMakeFiles/zig2.dir/zig2.c.o -c $SRC_DIR/build-release/zig2.c
$SRC_DIR/build-release/zig2.c:266757:16: error: conflicting types for 'sigaction'; have 'int(int,  const struct Sigaction__3652 *, struct Sigaction__3652 *)'
266757 | zig_extern int sigaction(int, struct Sigaction__3652 const *, struct Sigaction__3652 *);
       |                ^~~~~~~~~
In file included from $SRC_DIR/zig-source/stage1/zig.h:275,
                 from $SRC_DIR/build-release/zig2.c:2:
$BUILD_PREFIX/powerpc64le-conda-linux-gnu/sysroot/usr/include/signal.h:240:12: note: previous declaration of 'sigaction' with type 'int(int,  const struct sigaction * restrict,  struct sigaction * restrict)'
  240 | extern int sigaction (int __sig, const struct sigaction *__restrict __act,
      |            ^~~~~~~~~
ninja: build stopped: subcommand failed.

This is under a conda environment that installs sysroot for GLIBC 2.28 (which seems needed for zig). Is there a way to patch the source so that the built zig2.c complies with the /powerpc64le-conda-linux-gnu/sysroot/usr/include/signal.h? Or an alternative method to build zig for ppc64le?

Using the distributed binary fails on the Travis runner without any error (objdump -h works but nm fails)

Cross-compiling with CMake fails because qemu-ppc64le-static fails similarly when trying to execute /usr/bin/qemu-ppc64le-static -s 32097152 -d guest_errors -p 65536 zig-wasm2c

Cross-compiling with x86_64 binary distro also fails (I'd need to dig the loogs). But basically, I seem unable to build zig for PPC64LE

This is part of an effort to bring zig to the conda framework. Currently, x86_64, aarch64 and osx-64 builds with the recipe Thank you

Expected Behavior

completes the CMake build and generates a functional zig (the recipe tests the behavior.zig in an isolated and emulated environment

[MementoRC]

Update: Patching signal.h to comment out sigaction made the build continue. New Issue:

$SRC_DIR/build-release/zig2.c: In function 'os_linux_powerpc64_syscall5__7972':
$SRC_DIR/build-release/zig2.c:845006: error: 'asm' specifier for variable 't5' conflicts with 'asm' clobber list
845006 |  __asm volatile(" sc\n bns+ 1f\n neg 3, 3\n 1:": [ret]"=r"(t2): [number]"r"(t3), [arg1]"r"(t4), [arg2]"r"(t5), [arg3]"r"(t6), [arg4]"r"(t7), [arg5]"r"(t8): "memory", "cr0", "r4", "r5", "r6", "r7", "r8", "r9", "r10", "r11", "r12");
       | 
$SRC_DIR/build-release/zig2.c:845006: error: 'asm' specifier for variable 't6' conflicts with 'asm' clobber list
$SRC_DIR/build-release/zig2.c:845006: error: 'asm' specifier for variable 't7' conflicts with 'asm' clobber list
$SRC_DIR/build-release/zig2.c:845006: error: 'asm' specifier for variable 't8' conflicts with 'asm' clobber lis

There seems to be an collision with the definitions of the registers

The CMake cross-compilation failed due to a silly mistake, the zig-wasm2 used in COMMAND is in the binary, so when using the emulator the full path needs to be used

However, even though the emulation works the compilation sets the HOST_TARGET to linux-x64

[MementoRC]

Progressing on the cross-build till:

[3/4] Building C object CMakeFiles/zig2.dir/zig2.c.o
FAILED: CMakeFiles/zig2.dir/zig2.c.o 
/home/conda/feedstock_root/build_artifacts/zig_1718916935593/_build_env/bin/powerpc64le-conda-linux-gnu-cc -D_GNU_SOURCE -D__STDC_CONSTANT_MACROS -D__STDC_FORMAT_MACROS -D__STDC_LIMIT_MACROS -I/home/conda/feedstock_root/build_artifacts/zig_1718916935593/work/zig-source/stage1 -mcpu=power8 -mtune=power8 -ftree-vectorize -fPIC -fstack-protector-strong -fno-plt -O3 -pipe -isystem /home/conda/feedstock_root/build_artifacts/zig_1718916935593/_h_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placeho/include -fdebug-prefix-map=/home/conda/feedstock_root/build_artifacts/zig_1718916935593/work=/usr/local/src/conda/zig-0.13.0 -fdebug-prefix-map=/home/conda/feedstock_root/build_artifacts/zig_1718916935593/_h_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placeho=/usr/local/src/conda-prefix -O3 -DNDEBUG -std=c99 -O0 -fno-stack-protector -MD -MT CMakeFiles/zig2.dir/zig2.c.o -MF CMakeFiles/zig2.dir/zig2.c.o.d -o CMakeFiles/zig2.dir/zig2.c.o -c /home/conda/feedstock_root/build_artifacts/zig_1718916935593/work/build-release/zig2.c
{standard input}: Assembler messages:
{standard input}:9694643: Error: operand out of range (0xfffffffffdfffd90 is not between 0xfffffffffe000000 and 0x1fffffc)

I had to patch the zig2.c with args being in the clobber list

Somehow the compiler_rt.c has zig_f64 defined as a double instead of a long double and have several read 16bit from 8bits in memcpy()

[MementoRC]

The following instruction in building zig2: bl zig_shr_u32 gives: CMakeFiles/zig2.dir/zig2.c.s:9695561: Error: operand out of range (0xfffffffffdfff01c is not between 0xfffffffffe000000 and 0x1fffffc)

[MementoRC]

This is solved with -mlongcalls, but later the linker fails finalize zig2 with issues with libzigcpp.a

[MementoRC]

This is the PR trying to cross-cmake x86_64 -> ppc64le: https://dev.azure.com/conda-forge/feedstock-builds/_build/results?buildId=965648&view=logs&j=8c139a40-380b-5518-bf6b-edcb18cbf319&t=e71f8d0d-13be-528e-b5e7-bfc20e0db699&l=1001

It fails to make zig2 with tons of relocation issues that I don't know how to solve

[andrewrk]

Does it fail? From the log it looks like it timed out at Building zig2.

[MementoRC]

It does. The relocation errors is quite large so it is redirected to a file (not sure why it did not fail on the runner, it fails locally)

        echo "Building zig2"
        cmake --build . -v --target zig2 -- -j"${CPU_COUNT}" >> _build-zig2.log 2>&1

I don't understand the relocation errors (currently stuck there):

install
+- install zig
   +- zig build-exe zig ReleaseSmall powerpc64le-linux-gnu 170965 errors
error: ld.lld: /home/conda/feedstock_root/build_artifacts/zig_1719760384454/work/build-release/zigcpp/libzigcpp.a(zig_llvm.cpp.o):(function llvm::SmallVectorBase<unsigned int>::set_size(unsigned long) (.part.0): .text+0x28): unknown relocation (31) against symbol __assert_fail
error: ld.lld: /home/conda/feedstock_root/build_artifacts/zig_1719760384454/work/build-release/zigcpp/libzigcpp.a(zig_llvm.cpp.o):(function llvm::SmallVectorBase<unsigned int>::set_size(unsigned long) (.part.0): .text+0x38): unknown relocation (60) against symbol __assert_fail

[andrewrk]

Perhaps a question that can be answered by an LLD developer. Does LLD perhaps lack implementation of some relocations for ppc64le?

Or perhaps clang++ is emitting some strange relocations for that target, for some reason.

This target builds fine statically with zig-bootstrap, so maybe LLD is lacking a feature related to dynamic linking, or clang++ is lacking a feature related to dynamic linking.

[MementoRC]

So with zig-bootstrap (the 0.13.0 zig linux-x64 cross-targeting ppc64le):

+ /home/conda/feedstock_root/build_artifacts/zig_1719437002569/work/zig-bootstrap/zig build --prefix /home/conda/feedstock_root/build_artifacts/zig_1719437002569/_h_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placeho -Doptimize=ReleaseSafe -Dcpu=ppc64le -Dtarget=powerpc64le-linux-gnu -fqemu -Dconfig_h=/home/conda/feedstock_root/build_artifacts/zig_1719437002569/work/build-release/config.h -Denable-llvm -Dstrip -Duse-zig-libcxx=false -Dversion-string=0.13.0
install
+- install zig
   +- zig build-exe zig ReleaseSafe powerpc64le-linux-gnu 170965 errors
error: ld.lld: /home/conda/feedstock_root/build_artifacts/zig_1719437002569/work/build-release/zigcpp/libzigcpp.a(zig_clang.cpp.o):(function llvm::SmallVectorBase<unsigned int>::set_size(unsigned long) (.part.0): .text+0x28): unknown relocation (31) against symbol __assert_fail
error: ld.lld: /home/conda/feedstock_root/build_artifacts/zig_1719437002569/work/build-release/zigcpp/libzigcpp.a(zig_clang.cpp.o):(function llvm::SmallVectorBase<unsigned int>::set_size(unsigned long) (.part.0): .text+0x38): unknown relocation (60) against symbol __assert_fail
error: ld.lld: /home/conda/feedstock_root/build_artifacts/zig_1719437002569/work/build-release/zigcpp/libzigcpp.a(zig_clang.cpp.o):(function llvm::SmallVectorBase<unsigned int>::set_size(unsigned long) (.part.0): .text+0x3c): unknown relocation (119) against symbol __assert_fail

[Rexicon226]

The 31 and 60 relocs are valid, but they seem to not be implemented in LLD

https://refspecs.linuxfoundation.org/ELF/ppc64/PPC-elf64abi.html#RELOC

https://github.com/llvm/llvm-project/blob/main/lld/ELF/Relocations.h#L102-L106

[andrewrk]

The target powerpc64le-linux-gnu is dynamically linked glibc. A static build uses target powerpc64le-linux-musl and is the one that the CI tests with every commit. Looks like the dynamic glibc target will require more enhancements to LLD, or more progress on Zig's own linker (#17749). @kubkon is working on that.

[Rexicon226]

Mold does implement these relocs, maybe you could workaround by using it?

[andrewrk]

Zig does not integrate with mold since the plan is to provide our own linking functionality for this use case.

[MementoRC]

Very cool, I kept banging my head on a brick wall it seems. I don't follow very well, but are we saying that I can only create a statically built zig for ppc64le?

[andrewrk]

Yes, that is the current state of affairs, until Zig's linker surpasses LLD's functionality and adds these relocations.

[MementoRC]

Cool, I will try. Based upon your previous comment, I should focus on a CMake build, right. I do not have ppc64le, so I used a patched CMakeLists.txt with an emulator, do you see any issues with that as opposed to a native cmake build on Travis runner? They are quite a pain to use while debugging the recipe

I will open a different issue with the Windows build that is also not working at my end (suspecting something I don't understand again!)

[andrewrk]

For ppc64le specifically, I think your best bet is using https://github.com/ziglang/zig-bootstrap to cross-compile.

./build powerpc64le-linux-musl baseline

Be sure to read the README and use additional environment variables to utilize the logical cores of the host.

This build process will take a long time because it will

1. build llvm, clang, lld for the host
2. build zig for the host
3. use zig cc to build llvm, clang, lld for the target
4. build zig for the target

However, it is definitely working, because it is how the CI builds are produced.

[MementoRC]

Right, we would prefer to use the tools already available for host and target within the conda environment. We are basically trying to shortcut to 4 with LLVM, CLang, LLD available for both host/target - we also have zig on linux_64, do you see something that I am missing? Anyway, I will go back to it with your input and ping you back here if I have more questions - Thank you for the support

[MementoRC]

When working on the osx-arm64 cross-zig from osx-64, I get:

+ /Users/runner/miniforge3/conda-bld/zig_1719876422451/work/zig-bootstrap/zig build --prefix /Users/runner/miniforge3/conda-bld/zig_1719876422451/_h_env_ -Doptimize=ReleaseSafe -Dcpu=baseline --sysroot /Applications/Xcode_14.2.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX11.0.sdk -Dtarget=aarch64-macos-gnu -fqemu -Dconfig_h=/Users/runner/miniforge3/conda-bld/zig_1719876422451/work/build-release/config.h -Denable-llvm -Dstrip -Duse-zig-libcxx=false --libc /Users/runner/miniforge3/conda-bld/zig_1719876422451/work/_libc_file -Dversion-string=0.13.0
install
+- install zig
   +- zig build-exe zig ReleaseSafe aarch64-macos-gnu failure
error: error: unable to find or provide libc for target 'aarch64-macos.11.7.1...14.1-gnu'
info: zig can provide libc for related target aarch64-macos.11-none

error: the following command exited with error code 1:
/Users/runner/miniforge3/conda-bld/zig_1719876422451/work/zig-bootstrap/zig build-exe @/Users/runner/miniforge3/conda-bld/zig_1719876422451/work/zig-local-cache/args/2ee302878079fc9abd2f491d14cf120ea21be4098cdfbb8e49cf46450969aed3 
Build Summary: 276/279 steps succeeded; 1 failed (disable with --summary none)

I don't understand, I provide the sysroot and the libc file, and the target (I tried adding the .11 but that fails) I know it's unrelated to ppc64le, I could open a new Issue, if this is not proper

[andrewrk]

you passed aarch64-macos-gnu but you probably want aarch64-macos-none (please read the zig-bootstrap readme!)

[The-King-of-Toasters]

I recently gained access to the GCC compiler farm and I wanted test compatibility on a Power10 machine. Unfortunately I encountered many build errors that have limited me to using the 0.13 release via build-exe.

First, I tried building LLVM+Clang+LLD+Zig from source. The llvm portion went fine, but like @MementoRC mentioned at the top I kept running into conflicting type errors - more than sigaction. BTW, I also experienced the same conflicts on a Loongarch64 server which shows that it isnt ppc being special. I then tried using zig-bootstrap, but ran into the same errors.

Since I couldn't edit the system headers, I tried using the latest master build. Running zig init worked, but when I try to build anything I keep running into this error:

error: unable to build zig's multitarget libc: UnableToWriteArchive
error: unable to build compiler_rt: UnableToWriteArchive

Then I spent a lot of time doing bootstrap builds on an x86_64 server, but got to the same results:

• Build zig from source: C type conflicts in system headers.
• Zig bootstrap on ppc: Ditto
• Zig master build: UnableToWriteArchive.
• Zig 0.13 build: Linker errors due to posixMainAndExit, see #20087.
• Zig bootstrap on x86 -> ppc: posixMainAndExit error.
• Hacked zig bootstrap with latest zig sources: UnableToWriteArchive.

So, to build anything I have to run zig-0.13 build-exe -mcpu=[something-that-isn't-pwr10] main.zig. And I can't bisect because I can't build from source :(.

[The-King-of-Toasters]

Chatting with @alexrp led me to the same conclusion re: signals, but instead of patching out signal.h I've defined debugbreak it to be the instruction that GDB uses:

diff --git a/stage1/zig.h b/stage1/zig.h
index 05529feec7..38f01ca020 100644
--- a/stage1/zig.h
+++ b/stage1/zig.h
@@ -271,6 +271,8 @@ typedef char bool;
 #define zig_breakpoint() __asm__ volatile("bkpt #0");
 #elif defined(__aarch64__)
 #define zig_breakpoint() __asm__ volatile("brk #0");
+#elif defined(__powerpc__)
+#define zig_breakpoint() __asm__ volatile("twge r2,r2");
 #else
 #include <signal.h>
 #if defined(SIGTRAP)

With this, I get the same asm clobber errors mentioned above:

[2/4] Building C object CMakeFiles/zig2.dir/zig2.c.o
FAILED: CMakeFiles/zig2.dir/zig2.c.o
/usr/bin/cc -D_GNU_SOURCE -D__STDC_CONSTANT_MACROS -D__STDC_FORMAT_MACROS -D__STDC_LIMIT_MACROS -I/home/sg/gits/zig/stage1 -I/home/sg/local/llvm18-release/include -g -std=c99 -O0 -fno-stack-protector -MD -MT CMakeFiles/zig2.dir/zig2.c.o -MF CMakeFiles/zig2.dir/zig2.c.o.d -o CMakeFiles/zig2.dir/zig2.c.o -c /home/sg/gits/zig/build/zig2.c
/home/sg/gits/zig/build/zig2.c: In function ‘io_Writer_print__anon_63069__63069’:
/home/sg/gits/zig/build/zig2.c:901390: note: the ABI of passing aggregates with 16-byte alignment has changed in GCC 5
901390 | static uint16_t io_Writer_print__anon_63069__63069(struct io_Writer__9795 const a0, decl__63069_39 const a1) {
       |
/home/sg/gits/zig/build/zig2.c: In function ‘os_linux_powerpc64_syscall5__7851’:
/home/sg/gits/zig/build/zig2.c:816460: error: ‘asm’ specifier for variable ‘t5’ conflicts with ‘asm’ clobber list
816460 |  __asm volatile(" sc\n bns+ 1f\n neg 3, 3\n 1:": [ret]"=r"(t2): [number]"r"(t3), [arg1]"r"(t4), [arg2]"r"(t5), [arg3]"r"(t6), [arg4]"r"(t7), [arg5]"r"(t8): "memory", "cr0", "r4", "r5", "r6", "r7", "r8", "r9", "r10", "r11", "r12");
       |
/home/sg/gits/zig/build/zig2.c:816460: error: ‘asm’ specifier for variable ‘t6’ conflicts with ‘asm’ clobber list
/home/sg/gits/zig/build/zig2.c:816460: error: ‘asm’ specifier for variable ‘t7’ conflicts with ‘asm’ clobber list
/home/sg/gits/zig/build/zig2.c:816460: error: ‘asm’ specifier for variable ‘t8’ conflicts with ‘asm’ clobber list
/home/sg/gits/zig/build/zig2.c: In function ‘os_linux_powerpc64_syscall6__7852’:
/home/sg/gits/zig/build/zig2.c:1678944: error: ‘asm’ specifier for variable ‘t5’ conflicts with ‘asm’ clobber list
1678944 |  __asm volatile(" sc\n bns+ 1f\n neg 3, 3\n 1:": [ret]"=r"(t2): [number]"r"(t3), [arg1]"r"(t4), [arg2]"r"(t5), [arg3]"r"(t6), [arg4]"r"(t7), [arg5]"r"(t8), [arg6]"r"(t9): "memory", "cr0", "r4", "r5", "r6", "r7", "r8", "r9", "r10", "r11", "r12");
        |
/home/sg/gits/zig/build/zig2.c:1678944: error: ‘asm’ specifier for variable ‘t6’ conflicts with ‘asm’ clobber list
/home/sg/gits/zig/build/zig2.c:1678944: error: ‘asm’ specifier for variable ‘t7’ conflicts with ‘asm’ clobber list
/home/sg/gits/zig/build/zig2.c:1678944: error: ‘asm’ specifier for variable ‘t8’ conflicts with ‘asm’ clobber list
/home/sg/gits/zig/build/zig2.c:1678944: error: ‘asm’ specifier for variable ‘t9’ conflicts with ‘asm’ clobber list
/home/sg/gits/zig/build/zig2.c: In function ‘os_linux_powerpc64_syscall3__7849’:
/home/sg/gits/zig/build/zig2.c:1728823: error: ‘asm’ specifier for variable ‘t5’ conflicts with ‘asm’ clobber list
1728823 |  __asm volatile(" sc\n bns+ 1f\n neg 3, 3\n 1:": [ret]"=r"(t2): [number]"r"(t3), [arg1]"r"(t4), [arg2]"r"(t5), [arg3]"r"(t6): "memory", "cr0", "r4", "r5", "r6", "r7", "r8", "r9", "r10", "r11", "r12");
        |
/home/sg/gits/zig/build/zig2.c:1728823: error: ‘asm’ specifier for variable ‘t6’ conflicts with ‘asm’ clobber list
/home/sg/gits/zig/build/zig2.c: In function ‘os_linux_powerpc64_syscall4__7850’:
/home/sg/gits/zig/build/zig2.c:2324245: error: ‘asm’ specifier for variable ‘t5’ conflicts with ‘asm’ clobber list
2324245 |  __asm volatile(" sc\n bns+ 1f\n neg 3, 3\n 1:": [ret]"=r"(t2): [number]"r"(t3), [arg1]"r"(t4), [arg2]"r"(t5), [arg3]"r"(t6), [arg4]"r"(t7): "memory", "cr0", "r4", "r5", "r6", "r7", "r8", "r9", "r10", "r11", "r12");
        |
/home/sg/gits/zig/build/zig2.c:2324245: error: ‘asm’ specifier for variable ‘t6’ conflicts with ‘asm’ clobber list
/home/sg/gits/zig/build/zig2.c:2324245: error: ‘asm’ specifier for variable ‘t7’ conflicts with ‘asm’ clobber list
ninja: build stopped: subcommand failed.

@MementoRC can you share the changes you made to the clobber lists? I have no idea how inline asm works.

[MementoRC]

Sure, you can find the patches under: https://github.com/conda-forge/zig-feedstock/pull/30/files#diff-6c2d6aaa2d4b5f7aafb076ae683ca83a14d62b89abc619d56b3f586e60335c79

The last issue I encountered was with the relocation that are not supported by LLVM, but I need to build dynamically for the conda-forge recipe, this may not be an issue for a static build

[alexrp]

Just leaving a link to my comment over here: https://github.com/conda-forge/zig-feedstock/pull/44#issuecomment-2483558108

TL;DR: LLD doesn't support some relocations that GCC/binutils use in practice. Self-hosted linker will hopefully solve this.