Support nodejs >= 15, npm >= 7, lockfile version >= 2

infinisil commented 2 years ago

nodejs versions >= 15 use npm >= 7, which uses lockfile version 2 which is currently not supported by npmlock2nix, leading to failures like #139 and #140. See #146 for an attempt at implementing it. In addition, hook scripts, which npmlock2nix currently relies on, are also removed, see #110 and also #151 as a workaround.

PR #152 makes npmlock2nix fail early for nodejs version >= 15.

infinisil commented 2 years ago

In order to make this work, I'm thinking of these steps:

Introduce an easy way to patch arbitrary dependency sources, which to my knowledge can handle all use cases of hook scripts, but in a more manual way (you need to patch individual packages). This isn't ideal, but it's the best way forward. PR #151 implements this. This is fully backwards compatible.
Switch to using npm ci instead of npm install. npm install seems to not work with the new lockfile version, npm ci works though. This change then means that hook scripts don't work anymore and people will have to use the changes from 1. to do the changes that were previously done automatically. We should have some warning for this, because this is backwards incompatible.
Implement support for the new lockfile version and remove the early fail introduced in #152. This is fully backwards compatible.

Potential backwards-compatible alternatives to 2. are:

Since current npmlock2nix only relies on the preinstall hook script, it might be possible to replicate this behavior with 1. (#151) by automatically rewriting all sources. However, from some tests it seems like preinstall doesn't run on the raw source, unlike what #151 does, which makes this a bit more tricky. In addition, this would be rather wasteful, as every source would be in the store twice, often without any differences.
Use @milahu's npm-install-mini, which is a custom npm install written in NodeJS, that still implements hook scripts. See some arguments for this here. Disadvantage is an increased maintenance burden, an external dependency, and potential deviation from the standard npm.
Push for upstream npm to re-add something that could replace our use cases for hook scripts

cmoog commented 2 years ago

Is there an existing workaround for this issue? Now that node v16 is LTS, this is quite a wart in the side of nix <-> node compatibility.

cmoog commented 1 year ago

@andir @gilligan Is this project abandoned? I understand that the nix-commmunity org doesn't have the same guarantees as nixos, but at the moment this tool is completely incompatible with both the LTS and current version of node. It doesn't work with any active node version.

If so, I think a warning message in the README might be appropriate.

infinisil commented 1 year ago

@cmoog My first PR #151 towards fixing this issue was merged somewhat recently. I might have some time soon to continue with the next step from https://github.com/nix-community/npmlock2nix/issues/153#issuecomment-1090533718. Would love some feedback as to what you think the best approach to continue is.

mkhl commented 1 year ago

fwiw we're using this to build direnv/direnv-vscode and it's working just fine

flokli commented 1 year ago

With https://github.com/nix-community/npmlock2nix/pull/166 merged, and https://github.com/nix-community/npmlock2nix/issues/172 tracking the followup cleanup, this can be closed, no?

orangecms commented 1 year ago

Note: Node <15 is already obsolete. Node 16 will be EOL in 4 months. Current stable is Node 20.

I suggest closing this issue and leaving notes in the README that npmlock2nix needs maintenance.

c0deaddict commented 1 year ago

With #166 merged, and #172 tracking the followup cleanup, this can be closed, no?

I've updated to master but still get the error about the unsupported nodejs version, so it appears this isn't fixed with #166 merged?

flokli commented 1 year ago

Are you using npmlock2nix.v2?

c0deaddict commented 1 year ago

Are you using npmlock2nix.v2?

Aha, I wasn't, that does remove the warning, thanks! :+1: Although my build is now stuck in the nix builder on npm ci, probably trying to fetch something from the web, but that is a different issue :)

kevincox commented 1 year ago

It seems that nodejs 14 has been marked as insecure in nixpkgs-unstable and is now unsupported. So the current options are to use the insecure nodejs or use the beta v2.

Would be good to have a stable option here.

klarkc commented 1 year ago

@kevincox it's also happening on v16 (used by v2 api).

A workaround, when importing nixpkgs:

import inputs.nixpkgs {
  inherit system;
  # required by npmlock2nix
  config.permittedInsecurePackages = [ "nodejs-16.20.1" ];
}

nix-community / npmlock2nix

Support nodejs >= 15, npm >= 7, lockfile version >= 2 #153