Why are *.nix files not enough and why building the package requires package.json in current directory?

[mitar]

So I generated .nix files using node2nix --lock package-lock.json --nodejs-16 --composition node.nix --development and I expected that only .nix files are needed and all information from package-lock.json has been taken in. But to my surprise, if I run nix-build -E 'with import <nixpkgs> { }; callPackage ./default.nix { }' I get cryptic:

cp: missing destination file operand after '/nix/store/9yvclwmigxbii5iyw787bj5j691wnxvc-peerdb-search-package-json'
Try 'cp --help' for more information.

The reason is, package.json is needed at Nix package build time. That is a surprise. It is not defined as input anywhere in Nix packages. Why it is needed? Can this be documented somewhere? Have I missed something?

Example you can try running (it also does more stuff, like builds some Go stuff): https://gitlab.com/peerdb/search/-/jobs/3107194318/artifacts/browse

[svanderburg]

The reason that you need package.json is because node2nix invokes npm in the builder. The reason is that in addition to dependency management, npm also does build management, such as executing script directives. For that, a package.json file is still required.

[mitar]

But shouldn't then the file be embedded into a .nix file? Or at least this should be documented. I think embedding the file would also solve the "Updating the package lock file" issue.

Also, my understanding is that running npm install is optional and just a check, as written here? So maybe this should be changed into a CLI parameter, if you want to run npm install then package.json gets embedded into a .nix file and npm install is run, otherwise neither happen.

[mitar]

In fact, I think there is even a bigger problem here. First, this program creates nix file which copies whole current directory, recursively, to $out. This is probably a bit of overkill for most cases, but sure. (Maybe it should support some nixignore file or something.) Bigger problem is that all those files are not hash-controlled in any way. Where is sha256 of them in the nix file? They should be also stored (and cached) as one of inputs to the nix file.

For example buildGoModule has vendorSha256 and sha256 to control all the inputs. Something like that is missing here I think.

[milahu]

the error

installing
cp: missing destination file operand after '/nix/store/lq164q1qrcdq62bf6r5lflfjd2igx7xz-frida-compiler-agent-package-json'
Try 'cp --help' for more information.
error: builder for '/nix/store/1h64jsw7jyfivp23zwb3jqxyfv7hy138-frida-compiler-agent-package-json.drv' failed with exit code 1;

comes from

{
  nodeDependencies = nodeEnv.buildNodeDependencies (lib.overrideExisting args {
    src = stdenv.mkDerivation {
      name = args.name + "-package-json";
      src = nix-gitignore.gitignoreSourcePure [
        "*"
        "!package.json"
        "!package-lock.json"
      ] args.src;
      dontBuild = true;
      installPhase = "mkdir -p $out; cp -r ./* $out;";
    };
  });

and buildNodeDependencies

the installPhase could be more verbose

{
      installPhase = ''
        mkdir $out
        cp -v package.json $out
        [ -f package-lock.json ] && cp -v package-lock.json $out
      '';

now it says

installing
cp: cannot stat 'package.json': No such file or directory
error: builder for '/nix/store/v9f12mrwp4liv98zj87pys0skm7cbkpk-frida-compiler-agent-package-json.drv' failed with exit code 1;