Failing tests on Silicon Mac

[kamidev]

Test environment

➜  zig git:(master) sw_vers; clang --version; zig version
ProductName:        macOS
ProductVersion:        13.2
BuildVersion:        22D49
Homebrew clang version 15.0.7
Target: arm64-apple-darwin22.3.0
Thread model: posix
InstalledDir: /opt/homebrew/opt/llvm/bin
0.11.0-dev.6316+a5b34a61a
➜  zig-gamedev git:(main) ✗ git log -1 | tee
commit 0348e79b3927f6b4e3e635dbe0582cfa48efef36
Author: Michal Ziulek <michalziulek@gmail.com>
Date:   Mon Feb 6 20:26:24 2023 +0100

Updated build scripts to the latest Zig build API changes (part 3, all done except docs).

Steps to reproduce

➜  zig-gamedev git:(main) cd libs/zmath
➜  zmath git:(main) zig build test
Test [7/79] test.zmath.isNearEqual... SKIP
Test [17/79] test.zmath.min... FAIL (TestUnexpectedResult)
/Users/jonas/src/zig/zig/build/stage3/lib/zig/std/testing.zig:509:14: 0x104b9aaf3 in expect (test)
    if (!ok) return error.TestUnexpectedResult;
             ^
/Users/jonas/src/zig/zig-gamedev/libs/zmath/src/zmath.zig:806:9: 0x104ba0403 in test.zmath.min (test)
        try expect(v[1] == 1.0);
        ^
Test [18/79] test.zmath.max... FAIL (TestUnexpectedResult)
/Users/jonas/src/zig/zig/build/stage3/lib/zig/std/testing.zig:509:14: 0x104b9aaf3 in expect (test)
    if (!ok) return error.TestUnexpectedResult;
             ^
/Users/jonas/src/zig/zig-gamedev/libs/zmath/src/zmath.zig:847:9: 0x104ba0acf in test.zmath.max (test)
        try expect(v[1] == 1.0);
        ^
Test [23/79] test.zmath.clamp... FAIL (TestUnexpectedResult)
/Users/jonas/src/zig/zig/build/stage3/lib/zig/std/testing.zig:509:14: 0x104b9aaf3 in expect (test)
    if (!ok) return error.TestUnexpectedResult;
             ^
/Users/jonas/src/zig/zig-gamedev/libs/zmath/src/zmath.zig:1255:9: 0x104ba671f in test.zmath.clamp (test)
        try expect(approxEqAbs(v, f32x4(-100.0, 100.0, -100.0, 0.0), 0.0001));
        ^
Test [25/79] test.zmath.saturate... FAIL (TestUnexpectedResult)
/Users/jonas/src/zig/zig/build/stage3/lib/zig/std/testing.zig:509:14: 0x104b9aaf3 in expect (test)
    if (!ok) return error.TestUnexpectedResult;
             ^
/Users/jonas/src/zig/zig-gamedev/libs/zmath/src/zmath.zig:1297:9: 0x104ba6d3f in test.zmath.saturate (test)
        try expect(approxEqAbs(v, f32x4(0.0, 1.0, 0.0, 0.0), 0.0001));
        ^
Test [27/79] test.zmath.mod... SKIP
Test [38/79] test.zmath.length3... SKIP
72 passed; 3 skipped; 4 failed.

[kamidev]

After discussions on Discord, the most likely cause seem to be Zig miscompilations. No zig-gamedev sample apps seem to be affected, so our short-term fix is to skip these tests on macOS for now.

[Pyrolistical]

Are those zig miscompilations issues tracked in the ziglang repo?

[kamidev]

Are those zig miscompilations issues tracked in the ziglang repo?

No, we were waiting for the next zig release at the time. I'll check if they are still failing once zig-gamedev starts using zig 0.12.

[kamidev]

I tried the four zmath tests we skip on Apple Silicon again. Using zig ig-gamedev unstable branch and zig 0.13.0-dev.47+c231d9496, all four tests still fail.