sh: node-pre-gyp: command not found

[codygman]

Even after looking at your example here and the readme and the example from the readme:

{pkgs ? import <nixpkgs> {
    inherit system;
}, system ? builtins.currentSystem}:

let
  nodePackages = import ./default.nix {
    inherit pkgs system;
  };
in
nodePackages // {
  floomatic = nodePackages.floomatic.override {
    buildInputs = [ pkgs.pkgconfig pkgs.qt4 ];
  };
}

Using this override.nix:

{pkgs ? import <nixpkgs> {
    inherit system;
}, system ? builtins.currentSystem}:

let
  nodePackages = import ./default.nix {
    pkgs = import <nixpkgs> { inherit system; overlays = [ overlay ]; };
  };

  # force node2nix to fetch packages with our bearer token
  overlay = self: super: {
    fetchurl = opts: super.fetchurl (opts // {
      curlOpts = with super.lib; (toList (opts.curlOpts or [])) ++ [ "--header @${pkgs.writeText "headers.txt" "Authorization: my-bearer-token"}" ];

    });
  };

in nodePackages // {
  canvas = nodePackages.package.override {
    buildInputs = [ pkgs.node-pre-gyp ];
  };
}

Then building and specifying the package attribute with gives me:

$ nix-build override.nix -A package
... snip ...
Skipping 'fsevents' build as platform linux is not supported
...............] - : info lifecycle fsevents@1.2.13~install: fsevents@1.2.1[0m
> bcrypt@3.0.8 install /nix/store/lfg0735bwn2hzf78mawgynk622pj45qc-node_api-server-0.0.1/lib/node_modules/api-server/node_modules/bcrypt
> node-pre-gyp install --fallback-to-build

sh: node-pre-gyp: command not found
npm ERR! code ELIFECYCLE
npm ERR! syscall spawn
npm ERR! file sh
npm ERR! errno ENOENT
npm ERR! bcrypt@3.0.8 install: `node-pre-gyp install --fallback-to-build`
npm ERR! spawn ENOENT
npm ERR!
npm ERR! Failed at the bcrypt@3.0.8 install script.
npm ERR! This is probably not a problem with npm. There is likely additional logging output above.

npm ERR! A complete log of this run can be found in:
npm ERR!     /build/.npm/_logs/2020-07-16T23_23_23_722Z-debug.log

builder for '/nix/store/gf41jzmvcsrh6bvg3ds8m4l575i07a64-node_api-server-0.0.1.drv' failed with exit code 1
error: build of '/nix/store/gf41jzmvcsrh6bvg3ds8m4l575i07a64-node_api-server-0.0.1.drv' failed
... snip ...

[codygman]

A few problems with the above:

• I was overriding "canvas"
• I was overriding node-pre-gyp rather than providing it to bcrypt so it's install script can pass, like the dnschain example in the readme.

[codygman]

It seems that https://github.com/svanderburg/node2nix#adding-additionalglobal-npm-packages-to-a-packaging-process solved this.

[DamienCassou]

@codygman does everything work now for you? Can you please tell us what you did?

[svanderburg]

@codygman Maybe I could make the instructions for overriding a bit more clear. So in case you generate from a package.json then you need to override the package attribute. If you generate for a JSON file that defines an array, you must select the package identifier attribute.

[thomasjm]

@svanderburg I'm pretty confused by that question. When you generate from a JSON array, the docs show nice examples for how to override buildInputs for a specific package. But when you generate from package.json, the docs only demonstrate overriding the .package attribute, like this:

let ... in
nodePackages // {
  package = nodePackages.package.override {
    postInstall = "grunt";
  };
}

This makes it seem like it's impossible to override the buildInputs for individual packages when you use package.json, and that you can only do it for "all packages" -- is that right? I tried overriding buildInputs on package to provide a native dependency and the package that needs it didn't seem to pick it up...

[svanderburg]

Maybe I could clarify the override section a bit more -- a package generated from an attribute set as well as all packages derived from an array are overridable.

Maybe I should explain both scenarios to avoid future confusion.

[DerGuteMoritz]

I tried overriding buildInputs on package to provide a native dependency and the package that needs it didn't seem to pick it up...

@thomasjm In my case, this approach actually did the trick. Maybe there was another reason why your package didn't pick it up, e.g. perhaps it also needed pkgconfig or something?

[thomasjm]

@DerGuteMoritz I think the reason it wasn't working for me is I was using the shell attribute, so an override on package doesn't work. Now I'm using the newly added nodeDependencies attribute (from #199) and the same issue applies. But it turns out (unsurprisingly) you can override the attribute you're actually using and then it does seem to work.

FWIW, before I figured this out I edited default.nix to put the dependencies into globalBuildInputs, which also works, but is probably more heavy-handed.

@svanderburg: a bit more explanation in the docs about overriding would be very helpful. The ability to override something on an individual dependency package so that the override is reflected in shell, package, etc. when you're working with a package.json would also be nice.

[robintown]

After reading this issue, the README, and other examples, it is still unclear to me how to get dependencies like canvas to build. I have generated from a package.json, and am trying to override the build inputs like this:

nodePackages // {
  package = nodePackages.package.override {
    buildInputs = with pkgs; [
      pkgconfig
      pkgs.nodePackages.node-pre-gyp
      # other canvas dependencies
      cairo pango libpng libjpeg giflib librsvg
    ];
  };
}

This results in exactly the same sh: node-pre-gyp: command not found error when attempting to build canvas. I have tried replacing buildInputs with globalBuildInputs, and a number of other variations with no luck.

Edit: Also tried adding node-pre-gyp as a supplement, as was suggested in some issues (didn't work).

[robintown]

I finally got it working by passing the extra dependencies as globalBuildInputs to import ./node-packages.nix. For anyone else struggling with this issue, I give you my full default.nix for a project using Typescript and canvas:

let
  sources = import ./nix/sources.nix;
  pkgs = import sources.nixpkgs {};
  nodejs = pkgs."nodejs-14_x";
  nodeEnv = import ./nix/node-env.nix {
    inherit (pkgs) lib stdenv python2 runCommand writeTextFile;
    inherit pkgs nodejs;
    libtool = if pkgs.stdenv.isDarwin then pkgs.darwin.cctools else null;
  };
  nodePackages = import ./nix/node-packages.nix {
    inherit (pkgs) lib fetchurl nix-gitignore stdenv fetchgit;
    inherit nodeEnv;
    globalBuildInputs = with pkgs; [
      pkgs.nodePackages.typescript
      pkgs.nodePackages.node-pre-gyp
      pkgconfig
      cairo
      pango
      libpng
      libjpeg
      giflib
      librsvg
    ];
  };
in
  nodePackages // {
    package = nodePackages.package.override {
      postInstall = "tsc";
    };
  }

[TLATER]

The solution from @robintown makes sense to me, but this still requires overwriting the generated default.nix, which just doesn't seem like the right solution.

If I'm trying to call my package with the callPackage function as described in the readme:

prev.callPackage ./default.nix { };

I don't see a way to override the globalBuildInputs of the node-packages this is supposed to import.

[TLATER]

Ah, I see, overrides must be specified for the sub-component you will be using.

I.e., for the example described in the readme:

let
  nodeDependencies = (pkgs.callPackage ./default.nix {}).shell.nodeDependencies;
in

stdenv.mkDerivation {
  name = "my-webpack-app";
  src = ./my-app;
  buildInputs = [nodejs];
  buildPhase = ''
    ln -s ${nodeDependencies}/lib/node_modules ./node_modules
    export PATH="${nodeDependencies}/bin:$PATH"

    # Build the distribution bundle in "dist"
    webpack
    cp -r dist $out/
  '';
}

you would want something like:

let
  nodeDependencies = (pkgs.callPackage ({ pkgs, system }:
    let nodePackages = import ./default.nix { inherit pkgs system; };
    in nodePackages // {
      shell = nodePackages.shell.override {
        buildInputs = [ pkgs.nodePackages.node-gyp-build ];
      };
    }
    ) {}).shell.nodeDependencies;
    in

stdenv.mkDerivation {
  name = "my-webpack-app";
  src = ./my-app;
  buildInputs = [nodejs];
  buildPhase = ''
    ln -s ${nodeDependencies}/lib/node_modules ./node_modules
    export PATH="${nodeDependencies}/bin:$PATH"

    # Build the distribution bundle in "dist"
    webpack
    cp -r dist $out/
  '';
}

It's probably more ergonomic to - instead of including the override in this - create a separate override.nix file and import that instead of default.nix. Since this overrides default.nix, no generated files need to be edited.

[aidalgol]

I still get the node-gyp-build: command not found error. This is the order in which I have done things:

1. Run node2nix -14 -l (-14 because node2nix does not automatically use the correct node version based on the version specified in package.json)
2. Modify default.nix to add globalBuildInputs
3. Run nix-shell -A shell And I get sh: line 1: node-gyp-build: command not found

My default.nix does not look quite the same as @robintown, but I'm also not using TypeScript.

# This file has been generated by node2nix 1.9.0. Do not edit!

{pkgs ? import <nixpkgs> {
    inherit system;
  }, system ? builtins.currentSystem, nodejs ? pkgs."nodejs-14_x"}:

let
  nodeEnv = import ./node-env.nix {
    inherit (pkgs) stdenv lib python2 runCommand writeTextFile writeShellScript;
    inherit pkgs nodejs;
    libtool = if pkgs.stdenv.isDarwin then pkgs.darwin.cctools else null;
  };
in
import ./node-packages.nix {
  inherit (pkgs) fetchurl nix-gitignore stdenv lib fetchgit;
  inherit nodeEnv;
  globalBuildInputs = with pkgs; [
    pkgs.nodePackages.node-pre-gyp
    pkgconfig
  ];
}

EDIT: Adding pkgs.nodePackages.node-gyp-build seems to have resolved this particular issue, and now I have a different error, which I am fairly sure is unrelated, but I will leave here in case it is useful information for this issue:

sharp: Downloading https://github.com/lovell/sharp-libvips/releases/download/v8.11.3/libvips-8.11.3-linux-x64.tar.br
sharp: Installation error: getaddrinfo ENOTFOUND github.com
sharp: Please see https://sharp.pixelplumbing.com/install for required dependencies
npm ERR! code ELIFECYCLE
npm ERR! errno 1
npm ERR! sharp@0.29.3 install: `(node install/libvips && node install/dll-copy && prebuild-install) || (node install/can-compile && node-gyp rebuild && node install/dll-copy)`
npm ERR! Exit status 1

[TLATER]

That is indeed unrelated; I ran into that too when I originally stumbled across this. Sharp is trying to download a statically compiled version of libvips that they are hosting on their git repository. If I recall correctly, it will use a dynamically linked version if it can find one.

This is what my override.nix presumably doing the same thing looks like:

{ pkgs ? import <nixpkgs> { inherit system; }, system ? builtins.currentSystem
}:

let nodePackages = import ./default.nix { inherit pkgs system; };
in nodePackages // {
  shell = nodePackages.shell.override {
    buildInputs = with pkgs; [
      nodePackages.node-gyp-build
      pkgconfig

      # For sharp (dependency of the auto-favicon
      # generator). This depends on vips (which uses
      # gobjects), and builds using node-gyp, which in turn
      # needs python.
      vips
      glib
    ];
  };
}

[aidalgol]

Back to the original issue, specifying node-gyp-build in override.nix is not working for me. After running node2nix -14 -l, I crate an override.nix file with the following contents,

{ pkgs ? import <nixpkgs> { inherit system; }, system ? builtins.currentSystem
}:

let nodePackages = import ./default.nix { inherit pkgs system; };
in nodePackages // {
  shell = nodePackages.shell.override {
    buildInputs = with pkgs; [
      nodePackages.node-gyp-build
      pkgconfig
    ];
  };
}

Then run nix-shell -A shell, and eventually get the same sh: line 1: node-gyp-build: command not found error.

[TLATER]

Just in case you don't realize, override.nix will not magically override a default.nix. You will need to explicitly change any references to default.nix (or a directory containing a default.nix that implicitly calls it), calling override.nix instead.

The name is just a downstram convention to make those modifications in a reasonably encapsulated way, without manually modifying the auto-generated default.nix or creating an unreasonably tall directory structure just for call-site ergonomics. It's a completely normal nix file otherwise, that just happens to import the default.nix of the directory it is in, which admittedly is a bit confusing because that file is treated specially by nix.

[aidalgol]

I was not aware of that, thanks.