Cloudef / zig2nix

Flake for packaging, building and running Zig projects.
MIT License
83 stars 1 forks source link

Doesn't correctly fetch with `zon2json-lock` #14

Closed Arya-Elfren closed 4 months ago

Arya-Elfren commented 4 months ago

When running it on zfe:

nix run github:Cloudef/zig2nix#zon2json-lock
fetching (nix hash): git+https://github.com/rockorager/libvaxis#75ac36ca61999c2f29467e02016322551e98bbd7
fetching (nix hash): git+https://github.com/zigimg/zigimg#3a667bdb3d7f0955a5a51c8468eac83210c1439e
fetching (nix hash): git+https://github.com/ryleelyman/GapBuffer.zig#main
Initialized empty Git repository in /tmp/git-checkout-tmp-LeiVxBil/GapBuffer.zig-main/.git/
fatal: couldn't find remote ref refs/tags/main
From https://github.com/ryleelyman/GapBuffer.zig
 * branch            HEAD       -> FETCH_HEAD
fatal: Not a valid object name
Unrecognized git object type:
Unable to checkout refs/tags/main from https://github.com/ryleelyman/GapBuffer.zig

When running it on flow:

nix run github:Cloudef/zig2nix#zon2json-lock
fetching (nix hash): https://github.com/Hejsil/zig-clap/archive/c0193e9247335a6c1688b946325060289405de2a.tar.gz
fetching (nix hash): https://github.com/neurocyte/zig-tracy/archive/58999b786089e5319dd0707f6afbfca04c6340e7.tar.gz
fetching (nix hash): https://github.com/wolfpld/tracy/archive/refs/tags/v0.10.tar.gz
fetching (nix hash): https://github.com/neurocyte/dizzy/archive/455d18369cbb2a0458ba70be919cd378338d695e.tar.gz
fetching (zig fetch): https://github.com/neurocyte/thespian/archive/b82d73557966428ac7db1507bb5a4b800ac6207c.tar.gz
fetching (nix hash): https://github.com/neurocyte/thespian/archive/b82d73557966428ac7db1507bb5a4b800ac6207c.tar.gz
fetching (nix hash): https://github.com/neurocyte/asio/archive/b9c9c23ef2e6f11b6123535ec33e5a23ed0c59da.tar.gz
fetching (nix hash): https://github.com/neurocyte/flow-themes/releases/download/master-15e8cad1619429bf2547a6819b5b999510d5c1e5/flow-themes.tar.gz
fetching (zig fetch): https://github.com/neurocyte/flow-syntax/archive/d476a9a178369822b1d7f3df07e8115980fbbaa7.tar.gz
fetching (nix hash): https://github.com/neurocyte/flow-syntax/archive/d476a9a178369822b1d7f3df07e8115980fbbaa7.tar.gz
fetching (zig fetch): https://github.com/neurocyte/tree-sitter/releases/download/master-93fd2b9c4f082e0cf88eec21846977ba832957af/source.tar.gz
fetching (nix hash): https://github.com/neurocyte/tree-sitter/releases/download/master-93fd2b9c4f082e0cf88eec21846977ba832957af/source.tar.gz
fetching (nix hash): https://github.com/fjebaker/fuzzig/archive/0fd156d5097365151e85a85eef9d8cf0eebe7b00.tar.gz
fetching (zig fetch): https://github.com/rockorager/libvaxis/archive/40e51ad0542ea964190db411292660c274e2a984.tar.gz
fetching (nix hash): https://github.com/rockorager/libvaxis/archive/40e51ad0542ea964190db411292660c274e2a984.tar.gz
fetching (nix hash): git+https://github.com/zigimg/zigimg#3a667bdb3d7f0955a5a51c8468eac83210c1439e
fetching (nix hash): git+https://github.com/ryleelyman/GapBuffer.zig#main
Initialized empty Git repository in /tmp/git-checkout-tmp-CUtvrSeL/GapBuffer.zig-main/.git/
fatal: couldn't find remote ref refs/tags/main
From https://github.com/ryleelyman/GapBuffer.zig
 * branch            HEAD       -> FETCH_HEAD
fatal: Not a valid object name
Unrecognized git object type:
Unable to checkout refs/tags/main from https://github.com/ryleelyman/GapBuffer.zig.
Cloudef commented 4 months ago

I think it's a problem that vaxis is referencing a main branch for gapbuffer which will break if any commits are pushed to there. But I think zon2json-lock should still handle this case.

Cloudef commented 4 months ago

https://github.com/Cloudef/zig2nix/commit/77a253890d18a19f9d64ab1c330bdbc6eb5e4577 Should fix this. I also pinged @rockorager about the gapbuffer dep.

rockorager commented 4 months ago

Fixed in libvaxis here

Arya-Elfren commented 4 months ago

zon2json-lock now works on these repos, but building them doesn't as they still reference older commits than the fix above. I assume this just requires waiting for them to update the dependencies?

Cloudef commented 4 months ago

@Arya-Elfren can you share your work somewhere so I can check it out?

Arya-Elfren commented 4 months ago

hm, it seems like it's working when I try to reproduce it separately from my nixos config? (other than the zig compile error...) Anyway, I think it's fixed, thanks!

Cloudef commented 4 months ago

I could check it still in case if you want

Arya-Elfren commented 4 months ago

I wrote the below process whilst researching it, but I think the problem is just that zig master had a breaking build.zig change recently merged. Now just to find which commit so I can pin to the zig before that...

Which made me run nix run github:Cloudef/zig2nix/master#versions which errors with:

master
default
jq: error (at /nix/store/5rj4q356xpyrj5i41nmv4frjayy7r32g-versions.json:865): Invalid numeric literal at EOF at line 1, column 3 (while parsing 'git')

uh, ok yeah, both zfe and flow have the same compiler error which is suspicious.

This process worked for hevi but fails with the same compiler error for flow and zfe. I would naively guess that you're not making zig verify the download hash compared to the .zon hash?

/tmp/zconv > git clone git@github.com:BrookJeynes/zfe.git
Cloning into 'zfe'...
remote: Enumerating objects: 274, done.
remote: Counting objects: 100% (13/13), done.
remote: Compressing objects: 100% (10/10), done.
remote: Total 274 (delta 3), reused 6 (delta 3), pack-reused 261
Receiving objects: 100% (274/274), 65.04 KiB | 600.00 KiB/s, done.
Resolving deltas: 100% (173/173), done.
/tmp/zconv > cd zfe
/tmp/zconv/zfe main > nix run github:Cloudef/zig2nix/master#zon2json-lock
fetching (nix hash): git+https://github.com/rockorager/libvaxis#75ac36ca61999c2f29467e02016322551e98bbd7
fetching (nix hash): git+https://github.com/zigimg/zigimg#3a667bdb3d7f0955a5a51c8468eac83210c1439e
fetching (nix hash): git+https://github.com/ryleelyman/GapBuffer.zig#main
fetching (nix hash): git+https://github.com/jinzhongjia/znvim#515fb2784f7b0db2ea7a0021aada8d4aa89a08fe
fetching (nix hash): https://github.com/zigcc/zig-msgpack/archive/refs/tags/0.0.5.tar.gz
fetching (nix hash): git+https://codeberg.org/dude_the_builder/zg#6d2ba3ff8f3cb7c86cef2b2176fb5b4a72c104dc
fetching (nix hash): git+https://github.com/mitchellh/libxev#f6a672a78436d8efee1aa847a43a900ad773618b
fetching (nix hash): git+https://github.com/fjebaker/fuzzig#0fd156d5097365151e85a85eef9d8cf0eebe7b00
fetching (nix hash): git+https://github.com/KeithBrown39423/zuid#49e5980ba83f7d9ae967fa7ce4d54384c1c0f82b
/tmp/zconv/zfe main ?1 > kak flake.nix
/tmp/zconv/zfe main ?2 > git add .
/tmp/zconv/zfe main +2 > nix run .#zfe                                                                                                                                                                                14:50:39
warning: Git tree '/tmp/zconv/zfe' is dirty
warning: creating lock file '/tmp/zconv/zfe/flake.lock'
warning: Git tree '/tmp/zconv/zfe' is dirty
error: builder for '/nix/store/g75bapq4gkghmi1lybqc7bf65p7j12vi-zfe-0.4.0.drv' failed with exit code 2;
       last 10 log lines:
       > /nix/store/wp3lc963r8mzh7igs4xa3zyqa4q0xmim-zig-0.14.0-dev.364+8ab70f80c/lib/std/Progress.zig:80:18: note: struct declared here
       > /build/tmp.EAMLdgyT6J/p/12207b7a5b538ffb7fb18f954ae17d2f8490b6e3778a9e30564ad82c58ee8da52361/src/build/ScdocStep.zig:123:25: error: expected type '*const fn (*Build.Step, Build.Step.MakeOptions) anyerror!void', found '*const fn (*Build.Step, Progress.Node) @typeInfo(@typeInfo(@TypeOf(src.build.ScdocStep.InstallStep.make)).Fn.return_type.?).ErrorUnion.error_set!void'
       > /build/tmp.EAMLdgyT6J/p/12207b7a5b538ffb7fb18f954ae17d2f8490b6e3778a9e30564ad82c58ee8da52361/src/build/ScdocStep.zig:123:25: note: pointer type child 'fn (*Build.Step, Progress.Node) @typeInfo(@typeInfo(@TypeOf(src.build.ScdocStep.InstallStep.make)).Fn.return_type.?).ErrorUnion.error_set!void' cannot cast into pointer type child 'fn (*Build.Step, Build.Step.MakeOptions) anyerror!void'
       > /build/tmp.EAMLdgyT6J/p/12207b7a5b538ffb7fb18f954ae17d2f8490b6e3778a9e30564ad82c58ee8da52361/src/build/ScdocStep.zig:123:25: note: parameter 1 'Progress.Node' cannot cast into 'Build.Step.MakeOptions'
       > /nix/store/wp3lc963r8mzh7igs4xa3zyqa4q0xmim-zig-0.14.0-dev.364+8ab70f80c/lib/std/Build/Step.zig:71:25: note: struct declared here
       > /nix/store/wp3lc963r8mzh7igs4xa3zyqa4q0xmim-zig-0.14.0-dev.364+8ab70f80c/lib/std/Progress.zig:80:18: note: struct declared here
       > /build/tmp.EAMLdgyT6J/p/12207b7a5b538ffb7fb18f954ae17d2f8490b6e3778a9e30564ad82c58ee8da52361/src/build/ScdocStep.zig:160:36: error: expected type 'Build.Step.MakeOptions', found 'Progress.Node'
       > /nix/store/wp3lc963r8mzh7igs4xa3zyqa4q0xmim-zig-0.14.0-dev.364+8ab70f80c/lib/std/Progress.zig:80:18: note: struct declared here
       > /nix/store/wp3lc963r8mzh7igs4xa3zyqa4q0xmim-zig-0.14.0-dev.364+8ab70f80c/lib/std/Build/Step.zig:71:25: note: struct declared here
       > /nix/store/wp3lc963r8mzh7igs4xa3zyqa4q0xmim-zig-0.14.0-dev.364+8ab70f80c/lib/std/Build/Step.zig:228:32: note: parameter type declared here
       For full logs, run 'nix log /nix/store/g75bapq4gkghmi1lybqc7bf65p7j12vi-zfe-0.4.0.drv'.
/tmp/zconv/zfe main +2 !1 >

In the kak flake.nix I just paste in the following flake:

{
  inputs = {
    flake-parts.url = "github:hercules-ci/flake-parts";
    nixpkgs.url = "github:NixOs/nixpkgs/nixos-unstable-small";
    zfe.flake = false;
    zfe.url = "github:BrookJeynes/zfe";
    zig2nix.inputs.nixpkgs.follows = "nixpkgs";
    zig2nix.url = "github:Cloudef/zig2nix";
  };

  outputs = { self, ... }@inputs: inputs.flake-parts.lib.mkFlake { inherit inputs; } rec {
    systems = inputs.nixpkgs.lib.systems.flakeExposed;
    perSystem = { pkgs, config, system, ... }: {
      packages = let
        zenv = inputs.zig2nix.outputs.zig-env.${system} {
          zig = inputs.zig2nix.outputs.packages.${system}.zig.master.bin;
        };
        build-zig = src: lock:
          (pkgs.lib.genAttrs zenv.lib.allTargetTriples (target: zenv.packageForTarget target ({
            src = src; zigBuildZonLock = lock;
          }))).${zenv.lib.zigTripleFromString system};
      in {
        zfe = build-zig (pkgs.lib.cleanSource inputs.zfe) ./build.zig.zon2json-lock;
      };
    };
  };
}

The full logs from the build phase of nix log /nix/store/g75bapq4gkghmi1lybqc7bf65p7j12vi-zfe-0.4.0.drv are:

Running phase: buildPhase
zig build flags: -Doptimize=ReleaseSafe -Dtarget=x86_64-linux-gnu
warning: Encountered error: FileNotFound, falling back to default ABI and dynamic linker.
/build/tmp.EAMLdgyT6J/p/12207b7a5b538ffb7fb18f954ae17d2f8490b6e3778a9e30564ad82c58ee8da52361/src/build/ScdocStep.zig:40:14: error: expected type '*const fn (*Build.Step, Build.Step.MakeOptions) anyerror!void', found '*const >
/build/tmp.EAMLdgyT6J/p/12207b7a5b538ffb7fb18f954ae17d2f8490b6e3778a9e30564ad82c58ee8da52361/src/build/ScdocStep.zig:40:14: note: pointer type child 'fn (*Build.Step, Progress.Node) @typeInfo(@typeInfo(@TypeOf(src.build.Scdo>
/build/tmp.EAMLdgyT6J/p/12207b7a5b538ffb7fb18f954ae17d2f8490b6e3778a9e30564ad82c58ee8da52361/src/build/ScdocStep.zig:40:14: note: parameter 1 'Progress.Node' cannot cast into 'Build.Step.MakeOptions'
/nix/store/wp3lc963r8mzh7igs4xa3zyqa4q0xmim-zig-0.14.0-dev.364+8ab70f80c/lib/std/Build/Step.zig:71:25: note: struct declared here
/nix/store/wp3lc963r8mzh7igs4xa3zyqa4q0xmim-zig-0.14.0-dev.364+8ab70f80c/lib/std/Progress.zig:80:18: note: struct declared here
/build/tmp.EAMLdgyT6J/p/12207b7a5b538ffb7fb18f954ae17d2f8490b6e3778a9e30564ad82c58ee8da52361/src/build/ScdocStep.zig:123:25: error: expected type '*const fn (*Build.Step, Build.Step.MakeOptions) anyerror!void', found '*const>
/build/tmp.EAMLdgyT6J/p/12207b7a5b538ffb7fb18f954ae17d2f8490b6e3778a9e30564ad82c58ee8da52361/src/build/ScdocStep.zig:123:25: note: pointer type child 'fn (*Build.Step, Progress.Node) @typeInfo(@typeInfo(@TypeOf(src.build.Scd>
/build/tmp.EAMLdgyT6J/p/12207b7a5b538ffb7fb18f954ae17d2f8490b6e3778a9e30564ad82c58ee8da52361/src/build/ScdocStep.zig:123:25: note: parameter 1 'Progress.Node' cannot cast into 'Build.Step.MakeOptions'
/nix/store/wp3lc963r8mzh7igs4xa3zyqa4q0xmim-zig-0.14.0-dev.364+8ab70f80c/lib/std/Build/Step.zig:71:25: note: struct declared here
/nix/store/wp3lc963r8mzh7igs4xa3zyqa4q0xmim-zig-0.14.0-dev.364+8ab70f80c/lib/std/Progress.zig:80:18: note: struct declared here
/build/tmp.EAMLdgyT6J/p/12207b7a5b538ffb7fb18f954ae17d2f8490b6e3778a9e30564ad82c58ee8da52361/src/build/ScdocStep.zig:160:36: error: expected type 'Build.Step.MakeOptions', found 'Progress.Node'
/nix/store/wp3lc963r8mzh7igs4xa3zyqa4q0xmim-zig-0.14.0-dev.364+8ab70f80c/lib/std/Progress.zig:80:18: note: struct declared here
/nix/store/wp3lc963r8mzh7igs4xa3zyqa4q0xmim-zig-0.14.0-dev.364+8ab70f80c/lib/std/Build/Step.zig:71:25: note: struct declared here
/nix/store/wp3lc963r8mzh7igs4xa3zyqa4q0xmim-zig-0.14.0-dev.364+8ab70f80c/lib/std/Build/Step.zig:228:32: note: parameter type declared here
Cloudef commented 4 months ago

Thanks, I fixed the .#versions. I can reproduce your build issue. Not sure what's going on there, but I'll check around. Btw, you can simply call zenv.package in your flake, as that already defaults to the current system. Or alternatively zenv.packageForTarget system the zigTripleFromString is not required as package.nix calls it already implicitly.

Cloudef commented 4 months ago

Changing the zig version to zig = inputs.zig2nix.outputs.packages.${system}.zig."0.13.0".bin; seems to fix zfe. Probably you just need to pin the correct zig for flow too.

Anyhow, here's corrected flake

{
  inputs = {
    flake-parts.url = "github:hercules-ci/flake-parts";
    nixpkgs.url = "github:NixOs/nixpkgs/nixos-unstable-small";
    zfe.flake = false;
    zfe.url = "github:BrookJeynes/zfe";
    zig2nix.inputs.nixpkgs.follows = "nixpkgs";
    zig2nix.url = "github:Cloudef/zig2nix";
  };

  outputs = { self, ... }@inputs: inputs.flake-parts.lib.mkFlake { inherit inputs; } {
    systems = inputs.nixpkgs.lib.systems.flakeExposed;
    perSystem = { pkgs, system, ... }: {
      packages = let
        zenv = inputs.zig2nix.outputs.zig-env.${system} {
          zig = inputs.zig2nix.outputs.packages.${system}.zig."0.13.0".bin;
        };
        build-zig = src: zigBuildZonLock: zenv.package { inherit src zigBuildZonLock; };
      in {
        zfe = build-zig (pkgs.lib.cleanSource inputs.zfe) ./build.zig.zon2json-lock;
      };
    };
  };
}