search

ziglang / zig

General-purpose programming language and toolchain for maintaining robust, optimal, and reusable software.
https://ziglang.org
MIT License
35.09k stars 2.56k forks source link

unable to build zig from source on NetBSD #13680

Open nikkicoon opened 1 year ago

nikkicoon commented 1 year ago

Zig Version

commit 4e078941d032c6ac0e1ea3c85989fd5d9a8747ba

Steps to Reproduce and Observed Behavior

Thanks for fixing #13673 Now this fails with:

[ 98%] Built target zig1
[ 99%] Building stage2 object /usr/work/lang/zig/work/zig-0.10.0/zig2.o
./lib/std/os.zig:5300:17: error: querying for canonical path of a handle is unsupported on this host
        else => @compileError("querying for canonical path of a handle is unsupported on this host"),
                ^
./lib/std/fs.zig:60:38: error: container 'std.os' has no member called 'MAXNAMLEN'
    .netbsd, .openbsd, .solaris => os.MAXNAMLEN,
                                     ^
--- zig2.o ---
*** [zig2.o] Error code 1

make[2]: stopped in /usr/work/lang/zig/work/zig-0.10.0
1 error

make[2]: stopped in /usr/work/lang/zig/work/zig-0.10.0
--- CMakeFiles/zig2.dir/all ---
*** [CMakeFiles/zig2.dir/all] Error code 2

make[1]: stopped in /usr/work/lang/zig/work/zig-0.10.0
1 error

make[1]: stopped in /usr/work/lang/zig/work/zig-0.10.0
*** [all] Error code 2

make: stopped in /usr/work/lang/zig/work/zig-0.10.0
1 error

make: stopped in /usr/work/lang/zig/work/zig-0.10.0
*** Error code 2

Stop.

same in the local git checkout of zig.

Expected Behavior

it compiles

kubkon commented 1 year ago

Is the linked commit hash a typo? I don't see how it's relevant to the failure.

nikkicoon commented 1 year ago

It was build on this commit, but the only relevant commit is a27bfae0365e8d6d7da695a114497f97b4b20a43 I guess.

kubkon commented 1 year ago

Ah, I see. I guess I got confused as I expected output from zig version pasted plus the linked commit didn't touch any of the problematic code :-)

nikkicoon commented 1 year ago

yeah, zig doesn't build, so I got no zig version output. latest commit made more sense to me.

nikkicoon commented 1 year ago

13684 results in one fewer error in this step:

[ 99%] Building stage2 object /usr/work/lang/zig/work/zig-0.10.0/zig2.o
./lib/std/os.zig:5301:17: error: querying for canonical path of a handle is unsupported on this host
        else => @compileError("querying for canonical path of a handle is unsupported on this host"),
                ^
--- zig2.o ---
*** [zig2.o] Error code 1

make[2]: stopped in /usr/work/lang/zig/work/zig-0.10.0
1 error

make[2]: stopped in /usr/work/lang/zig/work/zig-0.10.0
--- CMakeFiles/zig2.dir/all ---
*** [CMakeFiles/zig2.dir/all] Error code 2

make[1]: stopped in /usr/work/lang/zig/work/zig-0.10.0
1 error

make[1]: stopped in /usr/work/lang/zig/work/zig-0.10.0
*** [all] Error code 2

make: stopped in /usr/work/lang/zig/work/zig-0.10.0
1 error

make: stopped in /usr/work/lang/zig/work/zig-0.10.0
*** Error code 2

Stop.
semarie commented 1 year ago

see https://github.com/ziglang/zig/issues/13565 for the querying for canonical path of a handle is unsupported on this host error

nikkicoon commented 1 year ago

thanks!

nikkicoon commented 1 year ago

I believe this might be fixed as of today, as I just run into an issue where I forgot to point to libgcc:

make install                                                                                                                                                                                                                       [2/401]
-- Configuring zig version 0.11.0-dev.1181+8d64e5283
-- Configuring done
-- Generating done
-- Build files have been written to: /home/nikita/re-src/zig/build
[  5%] Building CXX object CMakeFiles/zigcpp.dir/src/zig_llvm.cpp.o
[ 10%] Linking CXX static library zigcpp/libzigcpp.a
[ 40%] Built target zigcpp
[ 50%] Built target zig-wasm2c
[ 55%] Converting /home/nikita/re-src/zig/stage1/zig1.wasm to /home/nikita/re-src/zig/build/zig1.c
[ 60%] Building C object CMakeFiles/zig1.dir/zig1.c.o
[ 65%] Linking C executable zig1
[ 70%] Built target zig1
[ 75%] Running zig1.wasm to produce /home/nikita/re-src/zig/build/compiler_rt.c
[ 80%] Running zig1.wasm to produce /home/nikita/re-src/zig/build/zig2.c
[ 85%] Building C object CMakeFiles/zig2.dir/zig2.c.o
[ 90%] Building C object CMakeFiles/zig2.dir/compiler_rt.c.o
[ 95%] Linking CXX executable zig2
[ 95%] Built target zig2
[100%] Building stage3
Unable to determine path to libgcc_eh.so
error: RequiredLibraryNotFound
/home/nikita/re-src/zig/build.zig:657:9: 0x291ceb in addCxxKnownPath (build)
        return error.RequiredLibraryNotFound;
        ^
/home/nikita/re-src/zig/build.zig:592:17: 0x292daa in addCmakeCfgOptionsToExe (build)
                try addCxxKnownPath(b, cfg, exe, b.fmt("libgcc_eh.{s}", .{lib_suffix}), null, need_cpp_includes);
                ^
/home/nikita/re-src/zig/build.zig:276:13: 0x29966c in build (build)
            try addCmakeCfgOptionsToExe(b, cfg, exe, use_zig_libcxx);
            ^
/home/nikita/re-src/zig/lib/build_runner.zig:229:24: 0x2821f7 in runBuild (build)
        .ErrorUnion => try root.build(builder),
                       ^
/home/nikita/re-src/zig/lib/build_runner.zig:210:5: 0x27e984 in main (build)
    try runBuild(builder);
    ^
error: the following build command failed with exit code 1: 
/home/nikita/re-src/zig/zig-cache/o/f52925468fd99e4da3795365c580f94f/build /home/nikita/re-src/zig/build/zig2 /home/nikita/re-src/zig /home/nikita/re-src/zig/zig-cache /home/nikita/.cache/zig compile --zig-lib-dir /home/nikita/re-src/zig/l
ib -Dconfig_h=/home/nikita/re-src/zig/build/config.h -Denable-llvm -Dtarget=native -Dcpu=native -Dversion-string=0.11.0-dev.1181+8d64e5283
*** Error code 1

Stop.
make[2]: stopped in /home/nikita/re-src/zig/build
*** Error code 1

Stop.
make[1]: stopped in /home/nikita/re-src/zig/build
*** Error code 1

Stop.
make: stopped in /home/nikita/re-src/zig/build

pkgsrc locally, on the same commit:

=> Tool dependency mktools-[0-9]*: found mktools-20220614                                                              
=> Tool dependency cmake>=2.8.1nb1: found cmake-3.25.1                                                                 
=> Tool dependency cwrappers>=20150314: found cwrappers-20220403
=> Tool dependency checkperms>=1.1: found checkperms-1.12                                                              
=> Full dependency bash-[0-9]*: found bash-5.2.15                                                                      
=> Full dependency lld>=9.0.1: found lld-15.0.6                                                                        
=> Full dependency llvm>=15: found llvm-15.0.6
=> Full dependency clang>=15.0.5nb1: found clang-15.0.6                                                                
===> Checking for vulnerabilities in zig-0.10.0.8d64e52nb2                                                             
===> Overriding tools for zig-0.10.0.8d64e52nb2                                                                                                                                                                                                
===> Extracting for zig-0.10.0.8d64e52nb2                                                                              
===> Patching for zig-0.10.0.8d64e52nb2              
=> Applying pkgsrc patches for zig-0.10.0.8d64e52nb2                                                                   
===> Creating toolchain wrappers for zig-0.10.0.8d64e52nb2                                                             
===> Configuring for zig-0.10.0.8d64e52nb2                                                                             
=> Replacing bash interpreter in lib/libcxxabi/src/demangle/cp-to-llvm.sh.
WARNING: [replace-interpreter] Skipping non-existent file "lib/libcxxabi/src/demangle/cp-to-llvm.sh".
=> Fixing CMAKE_MODULE_PATH in CMakeLists.txt
=> Checking for portability problems in extracted files
-- The C compiler identification is GNU 10.4.0                                                                         
-- The CXX compiler identification is GNU 10.4.0                                                                       
-- Detecting C compiler ABI info                     
-- Detecting C compiler ABI info - done                                                                                
-- Check for working C compiler: /usr/work/lang/zig/work/.cwrapper/bin/gcc - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: /usr/work/lang/zig/work/.cwrapper/bin/c++ - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Configuring zig version 0.11.0
-- Found llvm: -lLLVM-15;-lrt;-lexecinfo;-lpthread;-lm;-lz;-lzstd;-lterminfo;-lxml2 (Required is at least version "15")  
-- Found clang: /usr/pkg/lib/libclang-cpp.so.15 (Required is at least version "15") 
-- Found lld: /usr/pkg/lib/liblldMinGW.a;/usr/pkg/lib/liblldELF.a;/usr/pkg/lib/liblldCOFF.a;/usr/pkg/lib/liblldWasm.a;/usr/pkg/lib/liblldMachO.a;/usr/pkg/lib/liblldCommon.a (Required is at least version "15") 
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD - Failed
-- Looking for pthread_create in pthreads                                                                                                                                                                                              [37/339]
-- Looking for pthread_create in pthreads - not found
-- Looking for pthread_create in pthread
-- Looking for pthread_create in pthread - found
-- Found Threads: TRUE  
-- Configuring done
-- Generating done
CMake Warning:
  Manually-specified variables were not used by the project:

    CMAKE_INSTALL_LIBDIR
    CMAKE_INSTALL_MANDIR
    DESTDIR

-- Build files have been written to: /usr/work/lang/zig/work/zig-8d64e5283a7cc51ece36e32b050ede6d7282d1b1
=> Rewrite cmake Dependencies files
===> Building for zig-0.10.0.8d64e52nb2
[  5%] Building CXX object CMakeFiles/zigcpp.dir/src/zig_llvm.cpp.o
[ 10%] Building C object CMakeFiles/zig-wasm2c.dir/stage1/wasm2c.c.o
[ 15%] Building CXX object CMakeFiles/zigcpp.dir/src/zig_clang.cpp.o
[ 20%] Building CXX object CMakeFiles/zigcpp.dir/src/zig_llvm-ar.cpp.o
[ 25%] Linking C executable zig-wasm2c
[ 25%] Built target zig-wasm2c
[ 30%] Building CXX object CMakeFiles/zigcpp.dir/src/zig_clang_driver.cpp.o
[ 35%] Converting /usr/work/lang/zig/work/zig-8d64e5283a7cc51ece36e32b050ede6d7282d1b1/stage1/zig1.wasm to /usr/work/lang/zig/work/zig-8d64e5283a7cc51ece36e32b050ede6d7282d1b1/zig1.c
[ 40%] Building C object CMakeFiles/zig1.dir/stage1/wasi.c.o
[ 45%] Building C object CMakeFiles/zig1.dir/zig1.c.o
[ 50%] Building CXX object CMakeFiles/zigcpp.dir/src/zig_clang_cc1_main.cpp.o
[ 55%] Building CXX object CMakeFiles/zigcpp.dir/src/zig_clang_cc1as_main.cpp.o
[ 60%] Building CXX object CMakeFiles/zigcpp.dir/src/windows_sdk.cpp.o
[ 65%] Linking CXX static library zigcpp/libzigcpp.a
[ 65%] Built target zigcpp
[ 70%] Linking C executable zig1
[ 70%] Built target zig1
[ 75%] Running zig1.wasm to produce /usr/work/lang/zig/work/zig-8d64e5283a7cc51ece36e32b050ede6d7282d1b1/zig2.c
[ 80%] Running zig1.wasm to produce /usr/work/lang/zig/work/zig-8d64e5283a7cc51ece36e32b050ede6d7282d1b1/compiler_rt.c
[ 90%] Building C object CMakeFiles/zig2.dir/zig2.c.o
[ 90%] Building C object CMakeFiles/zig2.dir/compiler_rt.c.o
[ 95%] Linking CXX executable zig2
[ 95%] Built target zig2
[100%] Building stage3
Unable to determine path to libgcc_eh.so
error: RequiredLibraryNotFound
/usr/work/lang/zig/work/zig-8d64e5283a7cc51ece36e32b050ede6d7282d1b1/build.zig:657:9: 0x291ceb in addCxxKnownPath (build)
        return error.RequiredLibraryNotFound;
        ^
/usr/work/lang/zig/work/zig-8d64e5283a7cc51ece36e32b050ede6d7282d1b1/build.zig:592:17: 0x292daa in addCmakeCfgOptionsToExe (build)
                try addCxxKnownPath(b, cfg, exe, b.fmt("libgcc_eh.{s}", .{lib_suffix}), null, need_cpp_includes);
                ^
/usr/work/lang/zig/work/zig-8d64e5283a7cc51ece36e32b050ede6d7282d1b1/build.zig:276:13: 0x29966c in build (build)
            try addCmakeCfgOptionsToExe(b, cfg, exe, use_zig_libcxx);
            ^
/usr/work/lang/zig/work/zig-8d64e5283a7cc51ece36e32b050ede6d7282d1b1/lib/build_runner.zig:229:24: 0x2821f7 in runBuild (build)
        .ErrorUnion => try root.build(builder),
                       ^
/usr/work/lang/zig/work/zig-8d64e5283a7cc51ece36e32b050ede6d7282d1b1/lib/build_runner.zig:210:5: 0x27e984 in main (build)
    try runBuild(builder);
    ^
error: the following build command failed with exit code 1: 
/usr/work/lang/zig/work/zig-8d64e5283a7cc51ece36e32b050ede6d7282d1b1/zig-cache/o/470ba8542d5d19f0e8eadac9ce7b82ec/build /usr/work/lang/zig/work/zig-8d64e5283a7cc51ece36e32b050ede6d7282d1b1/zig2 /usr/work/lang/zig/work/zig-8d64e5283a7cc51ec
e36e32b050ede6d7282d1b1 /usr/work/lang/zig/work/zig-8d64e5283a7cc51ece36e32b050ede6d7282d1b1/zig-cache /usr/work/lang/zig/work/.home/.cache/zig compile --zig-lib-dir /usr/work/lang/zig/work/zig-8d64e5283a7cc51ece36e32b050ede6d7282d1b1/lib 
-Dconfig_h=/usr/work/lang/zig/work/zig-8d64e5283a7cc51ece36e32b050ede6d7282d1b1/config.h -Denable-llvm -Drelease -Dstrip -Dtarget=native -Dcpu=native -Dversion-string=0.11.0
--- CMakeFiles/stage3 ---
*** [CMakeFiles/stage3] Error code 1

make[2]: stopped in /usr/work/lang/zig/work/zig-8d64e5283a7cc51ece36e32b050ede6d7282d1b1
1 error

make[2]: stopped in /usr/work/lang/zig/work/zig-8d64e5283a7cc51ece36e32b050ede6d7282d1b1
--- CMakeFiles/stage3.dir/all ---
*** [CMakeFiles/stage3.dir/all] Error code 2

make[1]: stopped in /usr/work/lang/zig/work/zig-8d64e5283a7cc51ece36e32b050ede6d7282d1b1
1 error

make[1]: stopped in /usr/work/lang/zig/work/zig-8d64e5283a7cc51ece36e32b050ede6d7282d1b1
*** [all] Error code 2

make: stopped in /usr/work/lang/zig/work/zig-8d64e5283a7cc51ece36e32b050ede6d7282d1b1
1 error

make: stopped in /usr/work/lang/zig/work/zig-8d64e5283a7cc51ece36e32b050ede6d7282d1b1
*** Error code 2

Stop.
make[1]: stopped in /usr/pkgsrc/lang/zig
*** Error code 1

Stop.
make: stopped in /usr/pkgsrc/lang/zig
andrewrk commented 1 year ago

@mikdusan reported success building from source on NetBSD in #13700. I'm not sure what was different on the system he tested vs yours.

Here's a clue though: that stack trace indicates that zig tried to find the location to libgcc_eh.so (needed because, alas, LLVM is written in C++) by executing c++ -print-file-name=libgcc_eh.so but it didn't produce a viable result.

I agree, however, that the original issue you filed here is resolved now.

nikkicoon commented 1 year ago

On NetBSD, we do not have a libgcc_eh.so, the unwinder is part of libgcc_s or libc in the case of the llvm userland. Should I move this to a new bugreport then?

(this is reproduceable both in my uncommitted pkgsrc lang/zig build from a current git commit, and building from a git checkout locally)

andrewrk commented 1 year ago

Can you try this patch? 

--- a/build.zig
+++ b/build.zig
@@ -589,7 +589,6 @@ fn addCmakeCfgOptionsToExe(
             },
             .netbsd => {
                 try addCxxKnownPath(b, cfg, exe, b.fmt("libstdc++.{s}", .{lib_suffix}), null, need_cpp_includes);
-                try addCxxKnownPath(b, cfg, exe, b.fmt("libgcc_eh.{s}", .{lib_suffix}), null, need_cpp_includes);
             },
             .dragonfly => {
                 try addCxxKnownPath(b, cfg, exe, b.fmt("libstdc++.{s}", .{lib_suffix}), null, need_cpp_includes);\
nikkicoon commented 1 year ago

That sort of progresses beyond that, but I believe that could be "default stack size of 4MB fails with various errors like SIGSEGV when building stage3":

[ 40%] Built target zigcpp
[ 50%] Built target zig-wasm2c
[ 70%] Built target zig1
[ 95%] Built target zig2
[100%] Building stage3
Semantic Analysis [1083] append... error: zig...
error: The following command terminated unexpectedly:
/home/nikita/re-src/zig/build/zig2 build-exe --stack 33554432 /home/nikita/re-src/zig/src/main.zig -lc /home/nikita/re-src/zig/build/zigcpp/libzigcpp.a /usr/pkg/lib/libclang-cpp.so.15 /usr/pkg/lib/liblldMinGW.a /usr/pkg/lib/liblldELF.a /usr/pkg/lib/liblldCOFF.a /usr/pkg/lib/liblldWasm.a /usr/pkg/lib/liblldMachO.a /usr/pkg/lib/liblldCommon.a -lLLVM-15 -lexecinfo -lz -lzstd -lterminfo -lxml2 /usr/lib/libstdc++.so --cache-dir /home/nikita/re-src/zig/zig-cache --global-cache-dir /home/nikita/.cache/zig --name zig --pkg-begin build_options /home/nikita/re-src/zig/zig-cache/options/mRIQ5BrP0TVMMNd23I4jCilMZa8nnUQm-5uKsb741I_ErAmu9-3q8P6VhQNnK-ea --pkg-end -I /usr/pkg/include -I /usr/pkg/include -L /usr/pkg/lib -L /usr/pkg/lib -I /usr/pkg/include -fno-build-id --zig-lib-dir /home/nikita/re-src/zig/lib --enable-cache 
[100%] Built target stage3
Install the project...
-- Install configuration: "Debug"
error: zig...
error: The following command terminated unexpectedly:
/home/nikita/re-src/zig/build/zig2 build-exe --stack 33554432 /home/nikita/re-src/zig/src/main.zig -lc /home/nikita/re-src/zig/build/zigcpp/libzigcpp.a /usr/pkg/lib/libclang-cpp.so.15 /usr/pkg/lib/liblldMinGW.a /usr/pkg/lib/liblldELF.a /usr/pkg/lib/liblldCOFF.a /usr/pkg/lib/liblldWasm.a /usr/pkg/lib/liblldMachO.a /usr/pkg/lib/liblldCommon.a -lLLVM-15 -lexecinfo -lz -lzstd -lterminfo -lxml2 /usr/lib/libstdc++.so --cache-dir /home/nikita/re-src/zig/zig-cache --global-cache-dir /home/nikita/.cache/zig --name zig --pkg-begin build_options /home/nikita/re-src/zig/zig-cache/options/mRIQ5BrP0TVMMNd23I4jCilMZa8nnUQm-5uKsb741I_ErAmu9-3q8P6VhQNnK-ea --pkg-end -I /usr/pkg/include -I /usr/pkg/include -L /usr/pkg/lib -L /usr/pkg/lib -I /usr/pkg/include -fno-build-id --zig-lib-dir /home/nikita/re-src/zig/lib --enable-cache
nikkicoon commented 1 year ago

When I increase the stack size for my user (default: stack (-s kbytes ) 4096) to 32MB it works (but that feels like a hack around an issue).

mikdusan commented 1 year ago

On NetBSD, we do not have a libgcc_eh.so, the unwinder is part of libgcc_s or libc in the case of the llvm userland. Should I move this to a new bugreport then?

(this is reproduceable both in my uncommitted pkgsrc lang/zig build from a current git commit, and building from a git checkout locally)

I see what happened here... I only tested with static LLVM. By using dynamic linkage LLVM, we'd need to fixup TOP/build.zig to adapt.

A couple of things...

I'm online at irc and discord if you experience further issues.

Oh, you will need to bump stack limit in your shell/env before a stage3 build. zig2 demands more than typical stack. I guess the linking options to create zig2 via cmake -Wl,-z,stack-size=0x10000000 (256 MiB) is ignored in netbsd executables just as it is ignored on linux?

mikdusan commented 1 year ago

14225 will likely solve this issue

nikkicoon commented 1 year ago

This solved the linking, but the stack size issue remains.

nikkicoon commented 1 year ago

14225 will likely solve this issue

the fix was in 0.10.1, right? because I'm puzzled about how the same issue still appears on NetBSD 9.0, which we still support.

https://nyftp.netbsd.org/pub/pkgsrc/packages/reports/2023Q1/NetBSD-9.0-x86_64/20230516.1659/zig-0.10.1nb1/build.log

mikdusan commented 1 year ago

@nikicoon,

[100%] Building stage3
/work/lang/zig/work/zig-0.10.1/lib/std/os.zig:5358:17: error: querying for canonical path of a handle is unsupported on this host
                @compileError("querying for canonical path of a handle is unsupported on this host");
                ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

referenced by:
    realpathZ: /work/lang/zig/work/zig-0.10.1/lib/std/fs.zig:1549:32
    realpath: /work/lang/zig/work/zig-0.10.1/lib/std/fs.zig:1524:20
    remaining reference traces hidden; use '-freference-trace' to see all reference traces

Yes, #14225 is in 0.10.1, but that build.log error indicates a different issue entirely. 0.10.1 does not include 9bcd48e40e272408879a442b21099ea5ab6b85b4 which removes os.realpath / os.getFdPath. NetBSD does not support F.GETPATH prior to 10.x. The referenced commit removes that usage, but tbh I cannot remember if more than 1 commit was required to remove it.

nikkicoon commented 1 year ago

alright, thanks. I'll look into what's needed to backport it if it takes us much longer than zig 0.11 release to figure out the llvm 16.0.2 breakage in some packages