svanderburg / node2nix

Generate Nix expressions to build NPM packages
MIT License
513 stars 98 forks source link

referencing node_modules rebuilds them each time #118

Open qknight opened 5 years ago

qknight commented 5 years ago

short version

we've been using: node2nix -8 -d -i package.json to generate:

and using these we reference only the node_modules with ${nodePackages}/lib/node_modules/nixcloud-monitoring-frontend/node_modules

but each time we evaluate this it rebuilds the nodePackages and this takes about 8 minutes.

are we doing something wrong or is that always the case when using node2nix?

long version

let
  system = builtins.currentSystem;
  nodePackages = (import ./node2nix_default.nix {
    inherit pkgs system;
  }).package;
  # bower dependencies
  bowerDeps = pkgs.buildBowerComponents {
    src = ./nixcloud.monitoring-frontend;
    name = "bower-deps";
    generated = ./bower.nix;
  };
in stdenv.mkDerivation rec {
  name = "nixcloud.monitoring-frontend";
  src = ./nixcloud.monitoring-frontend;

  buildInputs = [ pkgs.nodejs pkgs.nodePackages.npm pkgs.nodePackages.bower ];

  configurePhase = ''
    # clean up stateful directories
    rm -Rf bower_components node_modules

    # bower and node packages
    ln -s ${nodePackages}/lib/node_modules/nixcloud-monitoring-frontend/node_modules node_modules
    ln -s ${bowerDeps}/bower_components

    export PATH=${nodePackages}/lib/node_modules/nixcloud-monitoring-frontend/node_modules/.bin:$PATH

nixos-rebuild switch

time nixos-rebuild build
building Nix...
building the system configuration...
these derivations will be built:
  /nix/store/4h4zi65aq2g7s0acbac9f72xzsmab90r-node-nixcloud-monitoring-frontend-0.0.0.drv
  /nix/store/wi260k5k7hkg7ig9slcnswn1prkq8gwc-nixcloud.monitoring-frontend.drv
  /nix/store/4cg2bv0pbi6zm4jcjdg8ir6km59w7si3-nginx.conf.drv
  /nix/store/nvb52dm2a8crilp5na130x4zl508cfb1-unit-script-nginx-pre-start.drv
  /nix/store/x9hh6rls60vddgnnxwz3p095qznqijkv-unit-nginx.service.drv
  /nix/store/937vgrsfhz5w5mvglkij089zmbdn3as4-system-units.drv
  /nix/store/838dzxld2myxxrmfkks4linmgvb1dqhl-etc.drv
  /nix/store/6pxpvxqs6skaar06gd77zqjacqgrjqrz-nixos-system-status.nixcloud.io-18.09.0919505-nixcloud_3f9b07a.drv
...

summary

so far we have been using yarn2nix but as of https://github.com/moretea/yarn2nix/issues/12 we can't use it without fixing node-sass. node-sass has been a show-stopper for a few packages in the past.

qknight commented 5 years ago

we got it to 'not' recompile using: https://github.com/svanderburg/node2nix/issues/8#issuecomment-432138008

basically edit node-packages.nix and reference a dummy folder where we copied the package.json into.

  args = {
    name = "nixcloud-monitoring-frontend";
    packageName = "nixcloud-monitoring-frontend";
    version = "0.0.0";
    src = ./dummy;
    dependencies = [
      (sources."@ember-decorators/argument-0.8.21" // {
        dependencies = [
          sources."babel-plugin-filter-imports-1.1.2"
          sources."ember-cli-version-checker-2.1.2"
        ];
      })
qknight commented 5 years ago

maybe it makes sense to mention this in the README.md

paulyoung commented 5 years ago

@qknight @pukkamustard when I do this I get a "do not know how to unpack source archive" error. Did you run into that?

lionello commented 5 years ago

Maybe too late, but for others: @paulyoung ./dummy is a folder, not a file.

paulyoung commented 5 years ago

@lionello it’s been a while but I’m pretty sure I was using a dummy directory containing package.json

sebastianblunt commented 2 years ago

Since https://github.com/svanderburg/node2nix/pull/199 / https://github.com/svanderburg/node2nix/pull/205 there's a nodeDependencies exposed that ignores everything but package.json and package-lock.json. Using https://github.com/svanderburg/node2nix/tree/d5ea8f0c2e35257350fb893ad1c27d9ea28e9e8b#using-the-nodejs-environment-in-other-nix-derivations I was able to avoid this unnecessary rebuilding by switching out shell.nodeDependencies for nodeDependencies.