Issue with webpack/webpack-cli

[kwohlfahrt]

I'm having some trouble getting my project up and running. I have a composition.json which includes all of the CLI scripts (and peer dependencies) I need:

["npm", "webpack", "webpack-cli", "typescript", "tslint", "typescript-formatter"]

I run node2nix -8 --input composition.json. This generates default.nix and a few others. However, when I try to run webpack, I get the error below:

$ webpack
One CLI for webpack must be installed. These are recommended choices, delivered as separate packages:
 - webpack-cli (https://github.com/webpack/webpack-cli)
   The original webpack full-featured CLI.
 - webpack-command (https://github.com/webpack-contrib/webpack-command)
   A lightweight, opinionated webpack CLI.
We will use "npm" to install the CLI via "npm install -D".
Which one do you like to install (webpack-cli/webpack-command):

Trying to install webpack-cli from this prompt gives the error in NixOS/nixpkgs#42275.

Trying to run webpack-cli directly gives this output:

$ webpack-cli
/nix/store/hdrbyr538j0wkgzh48w46rynjq6xynfg-node-webpack-cli-3.0.8/lib/node_modules/webpack-cli/bin/cli.js:244
                                throw err;
                                ^

Error: Cannot find module 'webpack'
    at Function.Module._resolveFilename (module.js:547:15)
    at Function.Module._load (module.js:474:25)
    at Module.require (module.js:596:17)
    at require (/nix/store/hdrbyr538j0wkgzh48w46rynjq6xynfg-node-webpack-cli-3.0.8/lib/node_modules/webpack-cli/node_modules/v8-compile-cache/v8-compile-cache.js:159:20)
    at Object.<anonymous> (/nix/store/hdrbyr538j0wkgzh48w46rynjq6xynfg-node-webpack-cli-3.0.8/lib/node_modules/webpack-cli/bin/convert-argv.js:7:24)
    at Module._compile (/nix/store/hdrbyr538j0wkgzh48w46rynjq6xynfg-node-webpack-cli-3.0.8/lib/node_modules/webpack-cli/node_modules/v8-compile-cache/v8-compile-cache.js:178:30)
    at Object.Module._extensions..js (module.js:663:10)
    at Module.load (module.js:565:32)
    at tryModuleLoad (module.js:505:12)
    at Function.Module._load (module.js:497:3)
    at Module.require (module.js:596:17)
    at require (internal/module.js:11:18)
    at yargs.parse (/nix/store/hdrbyr538j0wkgzh48w46rynjq6xynfg-node-webpack-cli-3.0.8/lib/node_modules/webpack-cli/bin/cli.js:241:14)
    at Object.parse (/nix/store/hdrbyr538j0wkgzh48w46rynjq6xynfg-node-webpack-cli-3.0.8/lib/node_modules/webpack-cli/node_modules/yargs/yargs.js:552:18)
    at /nix/store/hdrbyr538j0wkgzh48w46rynjq6xynfg-node-webpack-cli-3.0.8/lib/node_modules/webpack-cli/bin/cli.js:219:8
    at Object.<anonymous> (/nix/store/hdrbyr538j0wkgzh48w46rynjq6xynfg-node-webpack-cli-3.0.8/lib/node_modules/webpack-cli/bin/cli.js:530:3)
    at Module._compile (module.js:652:30)
    at Object.Module._extensions..js (module.js:663:10)
    at Module.load (module.js:565:32)
    at tryModuleLoad (module.js:505:12)
    at Function.Module._load (module.js:497:3)
    at Function.Module.runMain (module.js:693:10)
    at startup (bootstrap_node.js:191:16)
    at bootstrap_node.js:612:3

I've tried both nixos-unstable and master branch versions of node2nix, with the same result.

[svanderburg]

I think the problem you're facing is similar to making tools such as grunt-cli, eslint or gulp work in a Nix-workflow.

In an ordinary NPM workflow, the idea is that you install the CLI package globally e.g. through npm install -g which just provide the minimum amount functionality to make a CLI tool work. The actual functionality is provided as NPM packages that should be installed as devDependencies with the development project.

This blog post describes how to work with such a workflow with grunt-cli but the concepts are the same with webpack-cli: http://sandervanderburg.blogspot.com/2016/09/simulating-npm-global-package.html

Could you give it a try? Provide webpack-cli as supplement JSON package and the remaining packages as devDependencies with a project's package.json. Then I think it should work.

[kwohlfahrt]

I've put ["webpack-cli"] in supplement.json. Then I run: nix run nixpkgs.nodePackages.node2nix --command node2nix --development -8 --supplement-input supplement.json

package.json contains the following:

{
  "name": "rest",
  "version": "0.0.1",
  "description": "Test project",
  "main": "index.js",
  "scripts": {
    "test": "echo \"Error: no test specified\" && exit 1"
  },
  "author": "Kai Wohlfahrt",
  "license": "GPL-3.0",
  "dependencies": {
    "@types/react": "^16.4.0",
    "@types/react-dom": "^16.0.6",
    "@types/react-redux": "^6.0.2",
    "immutable": "^3.8.2",
    "react": "^16.4.1",
    "react-dom": "^16.4.1",
    "react-redux": "^5.0.7",
    "redux": "^4.0.0"
  },
  "devDependencies": {
    "awesome-typescript-loader": "^5.1.1",
    "source-map-loader": "^0.2.3",
    "typescript": "^2.9.2",
    "webpack": "^4.12.0"
  }
}

Then, I try to nix run -f default.nix, and get the following error:

builder for '/nix/store/07kbdpfigi8km095hak3vk32swpza45d-node-webpack-cli-3.0.8.drv' failed with exit code 1; last 10 log lines:
  npm ERR! errno ENOENT
  npm ERR! syscall spawn
  npm ERR! commitizen@2.10.1 postinstall: `opencollective postinstall`
  npm ERR! spawn ENOENT
  npm ERR! 
  npm ERR! Failed at the commitizen@2.10.1 postinstall script.
  npm ERR! This is probably not a problem with npm. There is likely additional logging output above.

  npm ERR! A complete log of this run can be found in:
  npm ERR!     /tmp/nix-build-node-webpack-cli-3.0.8.drv-0/.npm/_logs/2018-06-23T17_44_21_976Z-debug.log
cannot build derivation '/nix/store/jfb115x5vwmsxx2avhlfanqjxqa15hgi-node-react-nodes-0.0.1.drv': 1 dependencies couldn't be built

I've also tried creating the following override.nix:

{ pkgs ? import <nixpkgs> {}, system ? builtins.currentSystem }:

let
  nodePackages = import ./default.nix {
    inherit pkgs system;
  };
in
nodePackages // {
  package = nodePackages.package.override {
    postInstall = "webpack";
  };
}

and then running nix-build on that, and i get a similar error:

> commitizen@2.10.1 postinstall /nix/store/ppdv8y5ar2krkb1pqfc67yd840x3ndm2-node-webpack-cli-3.0.8/lib/node_modules/webpack-cli/node_modules/commitizen
> opencollective postinstall

sh: opencollective: command not found
npm ERR! file sh
npm ERR! code ELIFECYCLE
npm ERR! errno ENOENT
npm ERR! syscall spawn
npm ERR! commitizen@2.10.1 postinstall: `opencollective postinstall`
npm ERR! spawn ENOENT
npm ERR!
npm ERR! Failed at the commitizen@2.10.1 postinstall script.
npm ERR! This is probably not a problem with npm. There is likely additional logging output above.

npm ERR! A complete log of this run can be found in:
npm ERR!     /tmp/nix-build-node-webpack-cli-3.0.8.drv-0/.npm/_logs/2018-06-23T18_05_42_861Z-debug.log
builder for '/nix/store/07kbdpfigi8km095hak3vk32swpza45d-node-webpack-cli-3.0.8.drv' failed with exit code 1
cannot build derivation '/nix/store/6blaydpb5q21zsj0l3a3l21xl0qmca7r-node-react-nodes-0.0.1.drv': 1 dependencies couldn't be built
error: build of '/nix/store/22nlw4sv5ifqam7m2rksbc9av977by47-node-shell-react-nodes-0.0.1.drv', '/nix/store/6blaydpb5q21zsj0l3a3l21xl0qmca7r-node-react-nodes-0.0.1.drv' failed

Strangely, commitizen is not installed anywhere in NPM on my Debian system, so I don't know why its being pulled in here.

[svanderburg]

Looks like commitizen dependency (that can be a direct or indirect dependency of the package you're trying to deploy) is trying to invoke the opencollective executable, which does not seem to exist in the builder environment (as can be seen by inspecting the 4th line of your error trace).

Can you try to add this package: https://github.com/opencollective/opencollective-cli to the supplement.json?

[kwohlfahrt]

Ah, commitizen is pulled in because the supplement.json packages were also installed in dev mode, whereas on the other system I had them installed in production mode.

Now I get the error below:

builder for '/nix/store/08n7qfzs8by2jp4545lb4z5fa75rf3sq-node-opencollective-1.0.3.drv' failed with exit code 1; last 10 log lines:
  minizlib@1.1.0 /nix/store/9gld17snc89j2cnsjhnlb5z43s6r2ss5-node-opencollective-1.0.3/lib/node_modules/opencollective/node_modules/fsevents/node_modules/minizlib
  babel-cli@6.24.1 /nix/store/9gld17snc89j2cnsjhnlb5z43s6r2ss5-node-opencollective-1.0.3/lib/node_modules/opencollective/node_modules/babel-cli
  babel-register@6.26.0 /nix/store/9gld17snc89j2cnsjhnlb5z43s6r2ss5-node-opencollective-1.0.3/lib/node_modules/opencollective/node_modules/babel-cli/node_modules/babel-register
  babel-core@6.26.3 /nix/store/9gld17snc89j2cnsjhnlb5z43s6r2ss5-node-opencollective-1.0.3/lib/node_modules/opencollective/node_modules/babel-cli/node_modules/babel-register/node_modules/babel-core
  minimist@0.0.8 /nix/store/9gld17snc89j2cnsjhnlb5z43s6r2ss5-node-opencollective-1.0.3/lib/node_modules/opencollective/node_modules/babel-cli/node_modules/minimist
  npm ERR! code ENOTCACHED
  npm ERR! request to https://registry.npmjs.org/babel-core failed: cache mode is 'only-if-cached' but no cached response available.

  npm ERR! A complete log of this run can be found in:
  npm ERR!     /tmp/nix-build-node-opencollective-1.0.3.drv-0/.npm/_logs/2018-06-27T11_13_32_353Z-debug.log
cannot build derivation '/nix/store/3681n4x18627rj522ryay73z7qx62khp-node-react-nodes-0.0.1.drv': 1 dependencies couldn't be built
[1 built (1 failed), 0.0 MiB DL]
error: build of '/nix/store/3681n4x18627rj522ryay73z7qx62khp-node-react-nodes-0.0.1.drv', '/nix/store/8z80glha73vjy3w1py16d9mg9vs43pc1-node-shell-react-nodes-0.0.1.drv' failed

I don't currently have a package-lock.json, since this is a fresh new project.

[svanderburg]

Maybe the error is caused by a bug in the dependency resolution algorithm in the last stable version.

Can you try installing the development version by cloning the Git repo and running:

$ nix-env -f release.nix -iA package.x86_64-linux

I am actually close to releasing a new version of node2nix, so that this problem will go away.

[svanderburg]

I released a new version of node2nix last weekend, so install the latest versions of the master or 18.03 branches of Nixpkgs.

[mredaelli]

I'm having the same problem. @kwohlfahrt where you successful?

[costrouc]

@mredaelli and for others that find this issue. I ran into the same problems shown above. My solution was to move the devDependencies into dependencies here are the relevant files and everything works great for me. I am new to npm development with nix so take my approach with a grain of salt.

package.json

{
    "name": "chris-ostrouchov-website",
    "version": "1.0.0",
    "description": "personal website",
    "main": "index.js",
    "repository": "git@gitlab.com:costrouc/chrisostrouchov.com.git",
    "author": "Chris Ostrouchov <chris.ostrouchov@gmail.com>",
    "license": "MIT",
    "private": true,
    "dependencies": {
        "font-awesome": "^4.7.0",
        "katex": "^0.10.0",
        "babel-core": "^6.0.0",
        "babel-loader": "^7.0.4",
        "babel-preset-env": "^1.6.1",
        "babel-preset-es2015": "^6.24.1",
        "css-loader": "^1.0.1",
        "file-loader": "^2.0.0",
        "style-loader": "^0.23.1",
        "webpack": "^4.25.0",
        "webpack-cli": "^3.1.0"
    }
}

suplement.json

[
    "webpack",
    "webpack-cli"
]

override.nix

{ pkgs ? import <nixpkgs> {}
, system ? builtins.currentSystem
}:

let
  nodePackages = import ./default.nix {
    inherit pkgs system;
  };
in
nodePackages // {
  package = nodePackages.package.override {
    postInstall = "webpack";
  };
}

Some will not be relevant for you. Bash commands that I ran to build.

node2nix --supplement-input supplement.json
nix-build override.nix -A package