Packaging latest version in nixpkgs: "error: illegal path references in fixed-output derivation"

[bjornfor]

Hi,

I tried updating sbomnix (1.4.5 -> 1.6.0) in nixpkgs but hit this weird issue:

$ nix-build -A sbomnix
these 2 derivations will be built:
  /nix/store/g9zk3n7zhj4bgb22llns0y2kh2d89nr1-source.drv
  /nix/store/0y11vlrxgs9zlaxfv6dm6dkbclc0ml6h-sbomnix-1.6.0.drv
building '/nix/store/g9zk3n7zhj4bgb22llns0y2kh2d89nr1-source.drv'...

trying https://github.com/tiiuae/sbomnix/archive/refs/tags/v1.6.0.tar.gz
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
100  163k    0  163k    0     0   329k      0 --:--:-- --:--:-- --:--:--  329k
unpacking source archive /build/v1.6.0.tar.gz
error: illegal path references in fixed-output derivation '/nix/store/g9zk3n7zhj4bgb22llns0y2kh2d89nr1-source.drv'
error: 1 dependencies of derivation '/nix/store/0y11vlrxgs9zlaxfv6dm6dkbclc0ml6h-sbomnix-1.6.0.drv' failed to build

Not sure if this is an issue with Nix, nixpkgs or sbomnix, but thought I should start documenting the issue somewhere.

[joinemm]

We are currently in the process of updating the sbomnix version in nixpkgs.

I ran into this same issue when using the wrong sha256 hash, or lib.fakeHash. Updating the hash to one I got from running nix flake prefetch github:tiiuae/sbomnix/refs/tags/v1.6.0 makes the build actually start.

But there are some new python dependencies added to sbomnix, that are not specified in the derivation. Most of these are available in nixpkgs so they are trivial to add, but dfdiskcache is not packaged in nixpkgs unfortunately.

[henrirosten]

I see this same problem, and I also don't fully understand why this happens. Nixpkgs sbomnix update draft PR for reference: https://github.com/NixOS/nixpkgs/pull/299429.

[henrirosten]

I think the problem is fixed with: https://github.com/NixOS/nixpkgs/pull/299429/commits/68cc92f58bb643f2bfa4a13cc3bd0b82c8034aac

The commit message tries to explain the root-cause:

Apparently nix does not like '/nix/store/...' references in the source
tarball. Sbomnix documentation includes a number of store references.
Since documentation is part of the sbmonix tarball as downloaded from
github, we need to remove all documentation files from inside the
tarball that might have '/nix/store' references in them. This commit
adds a `postFetch` phase to the sbomnix build to do just that.

[bjornfor]

How about using nukeReferences instead of removing documentation/files?

[henrirosten]

I wasn't aware such script exists. How would you call it in this case to target everything under doc and all .md files also oustide doc?

[bjornfor]

I'm not at a PC now, so I don't know/remember exactly how the tool works, but either it recurses itself from a starting path or you have to use "find" and explicitly give all paths to it.

[henrirosten]

Tried with something like:

python.pkgs.buildPythonApplication rec {
  pname = "sbomnix";
  version = "1.6.1";

  src = fetchFromGitHub {
    owner = "tiiuae";
    repo = "sbomnix";
    rev = "refs/tags/v${version}";
    hash = "";

    # Remove documentation as it contains references to nix store
    postFetch = ''
      find "$out" -type f \( -name '*.md' -o -name '*.svg' \) -exec echo '{}' \; -exec ${nukeReferences}/bin/nuke-refs '{}' \;
    '';
  };

It runs but clearly the store reference replace fails somewhere as the build still fails. I opted using rm instead which seems way simpler in this case.

[henrirosten]

Closing this issue as the nixpkgs PR got merged.

[bjornfor]

Related: https://discourse.nixos.org/t/illegal-path-references-in-fixed-output-derivation/44360