cache bug when trying to build paoda/zba on windows

paoda commented 1 year ago

Zig Version

0.11.0-dev.4183+32a175740

Steps to Reproduce and Observed Behavior

Follow the build instructions here
Note that when running zig build the project does not build.

This issue only occurs on x86_64-windows builds of Zig. As of the writing of this issue, paoda/zba builds fine on x86_64-linux and x86_64-macos (?)

Running zig build with a zig compiler built with -Doptimize=ReleaseSafe reveals:

❯ wine ../../open/zig-bootstrap/out/zig-x86_64-windows-gnu-native/zig.exe build
0124:fixme:file:NtLockFile Unimplemented yet parameter
error: Unexpected
/home/paoda/dev/open/zig-bootstrap/out/host/lib/zig/std/fs.zig:1422:23: 0x1401f3332 in createFileW (zig.exe.obj)
        try w.LockFile(
                      ^
/home/paoda/dev/open/zig-bootstrap/out/host/lib/zig/std/fs.zig:1413:28: 0x1401f3344 in createFileW (zig.exe.obj)
        errdefer file.close();
                           ^
/home/paoda/dev/open/zig-bootstrap/out/host/lib/zig/std/fs.zig:1413:28: 0x1401f3358 in createFileW (zig.exe.obj)
        errdefer file.close();
                           ^
/home/paoda/dev/open/zig-bootstrap/out/host/lib/zig/std/fs.zig:1282:13: 0x140083999 in createFile (zig.exe.obj)
            return self.createFileW(path_w.span(), flags);
            ^
/home/paoda/dev/open/zig-bootstrap/out/host/lib/zig/std/Build/Cache.zig:429:29: 0x1400ab990 in hit (zig.exe.obj)
                else => |e| return e,
                            ^
/home/paoda/dev/open/zig-bootstrap/zig/src/Compilation.zig:1919:60: 0x1401085da in update (zig.exe.obj)
    defer if (comp.whole_cache_manifest != null) man.deinit();
                                                           ^
/home/paoda/dev/open/zig-bootstrap/zig/src/main.zig:3891:24: 0x140122989 in updateModule (zig.exe.obj)
    defer errors.deinit(comp.gpa);
                       ^
/home/paoda/dev/open/zig-bootstrap/zig/src/main.zig:4549:44: 0x1401c6b41 in cmdBuild (zig.exe.obj)
        const term = try child.spawnAndWait();
                                           ^
/home/paoda/dev/open/zig-bootstrap/zig/src/main.zig:213:5: 0x1400361f9 in main (zig.exe.obj)
    return mainArgs(gpa, arena, args);
    ^
/home/paoda/dev/open/zig-bootstrap/zig/src/main.zig:213:5: 0x140035c11 in main (zig.exe.obj)
    return mainArgs(gpa, arena, args);
    ^

The trace seems to suggest that the issue is with Cache.hit in Cache.zig: https://github.com/ziglang/zig/blob/23a806102a5a3d5b28b2e5ab5ec30e191daea6f4/lib/std/Build/Cache.zig#L408-L430

self.cache.manifest_dir.createFile receives the string ad148f9c52421264130d384a621c2e1e.txt which results in the function returning error.Unexpected instead of presumably creating the file.

Unfortunately I'm unfamiliar with the build system, so I'm unsure where to proceed next. However, if this is a contributor-friendly issue I'd love for the opportunity to fix this.

Expected Behavior

zig build produces a working zig binary

paoda commented 1 year ago

https://github.com/ziglang/zig/blob/98c7aec4e4ae7deac5e802a8c592c2d91b6b77af/lib/std/os/windows.zig#L2435-L2441

whoops, seems like I should figure out what the NTSTATUS value is next.

paoda commented 1 year ago

So NTSTATUS is 0xC0000002 suggesting that the reason the build is failing under wine is separate from the reason why the build is failing on Windows. Given the status code I imagine the wine issue to be something reporting upstream.

I just built a debug version of the Zig compiler (turns out I can't use zig-bootstrap eh?). Below is the refernce trace:

➜ C:\Users\paoda\dev\open\zig\stage4\bin\zig.exe build
thread 20320 panic: reached unreachable code
C:\Users\paoda\dev\open\zig\build\stage3\lib\zig\std\os\windows.zig:130:37: 0x7ff644508e31 in OpenFile (zig.exe.obj)
            .OBJECT_NAME_INVALID => unreachable,
                                    ^
C:\Users\paoda\dev\open\zig\build\stage3\lib\zig\std\fs.zig:1399:46: 0x7ff6442f9367 in createFileW (zig.exe.obj)
            .handle = try os.windows.OpenFile(sub_path_w, .{
                                             ^
C:\Users\paoda\dev\open\zig\build\stage3\lib\zig\std\fs.zig:1282:36: 0x7ff644132c15 in createFile (zig.exe.obj)
            return self.createFileW(path_w.span(), flags);
                                   ^
C:\Users\paoda\dev\open\zig\build\stage3\lib\zig\std\tar.zig:136:46: 0x7ff64426e4c1 in pipeToFileSystem__anon_55149 (zig.exe.obj)
                var file = try dir.createFile(file_name, .{});
                                             ^
C:\Users\paoda\dev\open\zig\src\Package.zig:603:33: 0x7ff64426f300 in unpackTarball__anon_54904 (zig.exe.obj)
    try std.tar.pipeToFileSystem(out_dir, decompress.reader(), .{
                                ^
C:\Users\paoda\dev\open\zig\src\Package.zig:519:30: 0x7ff6442788ce in fetchAndUnpack (zig.exe.obj)
            try unpackTarball(gpa, &req, tmp_directory.handle, std.compress.gzip);
                             ^
C:\Users\paoda\dev\open\zig\src\Package.zig:282:39: 0x7ff6442817e8 in fetchAndAddDependencies (zig.exe.obj)
        const sub = try fetchAndUnpack(
                                      ^
C:\Users\paoda\dev\open\zig\src\main.zig:4438:67: 0x7ff644286903 in cmdBuild (zig.exe.obj)
            const fetch_result = build_pkg.fetchAndAddDependencies(
                                                                  ^
C:\Users\paoda\dev\open\zig\src\main.zig:285:24: 0x7ff6440b3fdd in mainArgs (zig.exe.obj)
        return cmdBuild(gpa, arena, cmd_args);
                       ^
C:\Users\paoda\dev\open\zig\src\main.zig:213:20: 0x7ff6440b158e in main (zig.exe.obj)
    return mainArgs(gpa, arena, args);
                   ^
C:\Users\paoda\dev\open\zig\build\stage3\lib\zig\std\start.zig:507:101: 0x7ff6440b123c in main (zig.exe.obj)
    return @call(.always_inline, callMainWithArgs, .{ @as(usize, @intCast(c_argc)), @as([*][*:0]u8, @ptrCast(c_argv)), envp });
                                                                                                    ^
???:?:?: 0x7ff64609a0e4 in ??? (zig.exe)
???:?:?: 0x7ff64609a13b in ??? (zig.exe)
???:?:?: 0x7ffef28a7613 in ??? (KERNEL32.DLL)
???:?:?: 0x7ffef29e26b0 in ??? (ntdll.dll)

https://github.com/ziglang/zig/blob/9b56c7de79ee5aa4540f5427a37fd8883d1fb4e6/lib/std/tar.zig#L136-L137

This call to Dir.createFile fails because it is passed the string "tests/fuzzing/id:000000,sig:06,src:000024,time:10060,execs:1850,op:havoc,rep:4" which is the name of a file in the TOML parsing library I depend on

As a sanity check I tried cloning mattyhall/tomlz on Windows and as expected it wasn't successful due to the name of this file.

So in the end it wasn't really a Zig issue. All I can say is that I would hope the error be better communicated to the user but I imagine that not to be a high priority right now. 😅

ziglang / zig