[Feature]: provide a standard way to install yarn that uses configured nodejs

[ahobson]

Is your feature request related to a problem you're trying to solve? Please describe.

I'd like a way to install yarn that uses the version of nodejs configured in devbox.json.

I'm not sure if this could be considered a documentation bug or not, but the documented way of installing yarn does not use the configured version of node, which can lead in some pretty surprising results.

$ devbox version  
0.7.0
$ devbox init
$ devbox add nodejs@20.8 yarn
# ...
$ devbox run node --version
v20.8.0
$ devbox run yarn node --version
yarn node v1.22.19
v18.18.0
✨  Done in 0.03s.
$ devbox rm yarn
$ devbox add nodePackages.yarn
# ...
$ devbox run yarn node --version
yarn node v1.22.19
v18.18.0
✨  Done in 0.03s.

Describe the solution you'd like

Ideally it would be the default when installing the yarn package, but that's not the way nix works under the hood. I don't know if devbox could create/replace some shims since that would alter the nix behavior.

Describe alternatives you've considered

As a workaround, I've set up environment variables for a global npm installation:

  "env": {
    "NPM_CONFIG_PREFIX": "$PWD/.npmglobal",
    "PATH": "$PWD/node_modules/.bin:$PWD/.npmglobal/bin:$PATH"
  },

That works, but requires the user to npm install -g yarn first.

Additional context Add any other context or screenshots about the feature request here. Has the feature been requested before? If so, provide a link to the issue.

[ahobson]

It looks like with nix something like

(yarn.override { nodejs = pkgs.nodejs-20_x; })

would do the trick. I wonder if there's a way to allow that via a devbox option?

[empjustine]

Facing this exact same issue as well.

I'm using the workaround npx -p yarn@${my-version} yarn node --version that I was using before back when I was attempting to manage nodejs versions manually using pnpm env use --global 16.16.0 && npx -p yarn@${my-version} yarn node --version, but this is not entropy-resistant, doesn't work offline and isn't really locked and reproducible.

[empjustine]

I created a flake package ( github:empjustine/nodejs-flake/main#node-ad23450bfd78cc56739fef6150f95fb9a4105c4b33affe529fc0b71884cb2e12 at https://github.com/empjustine/nodejs-flake ) where I unceremoniously dump naive (read: nix-philosophy-offending, FHS compliant, environment dependant, default) npm installed packages.

There are three main problems.

1. This only really works if everything is x86_64-linux. In fact, this is hardcoded both in the package, the package dependencies, and the fact that I only have 1 build environment (my own computer).
2. This package only really works for this very specific set of npm/yarn dependencies I need on this project. Further projects with further dependencies will require other packages with their specific versions baked-in, compounding the problem.
3. Nixlang and nixpkgs knowledge definitely required for this, defeating the philosophy of devbox.

[jay-aye-see-kay]

My current work around is installing yarn like this in devbox.json.

{
  "packages": ["nodejs_18", "nodejs_18.pkgs.yarn"]
}

It seems to work well as long as both packages are installed and updated at the same time. I don't think this would work for arbitrary versions of node, but works well for the major versions available in nixpkgs.

I also think this deserves a bit of documentation because it's a really easy mistake to make that's very hidden and confusing to people unfamiliar with nix.

[empjustine]

I really need some very old, deprecated and odd versions of yarn and npm.

I remade my previously extremely awful flake to a sightly less sorry (still somewhat fhs compliant therefore cursed nix-abhorrent) state: github:empjustine/nodejs-flake/v2#npm-9_9_2.

This now dumps the npmjs tgz into the nix store and npm installs that tgz so that you don't need to totally trust the flake repository that much; just the npmjs packages.

It still only supports x86_64-linux but that is the only bultins.currentSystem I have to test this.

Unlike my (distant) inspiration https://github.com/nix-community/napalm, this doesn't have any fancy package.json or npm package processing, instead relying that the user will lock the nodejs itself and that it's environment/devbox $PATH and /usr/bin/env node points to somewhere sane and locked.

This won't solve the weird states where you really need to run mixed combinations of libc/yarn/npm/nodejs versions, say, to reliably reproduce issues that happens when CI, the test container and inside the production container contains mismatched versions of libc/yarn/npm/nodejs, but for those cases you need to atomically swap the entire devbox.json+devbox.lock to get same behavior anyways.

[hlubek]

I found https://github.com/jetpack-io/devbox/tree/main/examples/flakes/overlay which fixes it in a clean way. Although it's a bit more complicated when upgrading the Node.js version, since the commit hash / package name in the local Flake has to be adjusted accordingly (also in flake.lock by deleting it and calling devbox update).

[hlubek]

I found https://github.com/jetpack-io/devbox/tree/main/examples/flakes/overlay which fixes it in a clean way. Although it's a bit more complicated when upgrading the Node.js version, since the commit hash / package name in the local Flake has to be adjusted accordingly (also in flake.lock by deleting it and calling devbox update).

Here's an example of flake.nix that works for nodejs@16.20.2:

# We want to assure that "yarn node --version" is equal to "node --version".
# Copied from https://github.com/jetpack-io/devbox/blob/main/examples/flakes/overlay/yarn-overlay/flake.nix .
# See also https://github.com/jetpack-io/devbox/issues/1577 .
{
  description =
    "This flake outputs a modified version of Yarn that uses NodeJS 16";

  inputs = {
    # Note: This commit hash corresponds to the Node.js pkg commit hash in devbox.lock !!!
    nixpkgs.url = "nixpkgs/a71323f68d4377d12c04a5410e214495ec598d4c";
    flake-utils.url = "github:numtide/flake-utils";
  };

  outputs = { self, nixpkgs, flake-utils }:
  # Use the flake-utils lib to easily create a multi-system flake
  flake-utils.lib.eachDefaultSystem (system:
    let
      # You can define overlays as functions using the example below
      # This overlay will modify yarn to use nodejs_16
      overlay = (final: prev: {
        yarn = prev.yarn.override { nodejs = final.pkgs.nodejs_16; };
      });

      #
      pkgs =
        import nixpkgs {
          # Note: this needed to be added to allow an "insecure" (i.e. end of life) version of Node.js, since Nix is so very secure.
          config.permittedInsecurePackages = [
            "nodejs-16.20.2"
          ];

          inherit system;
          # Add your overlays to the list below. Note that they will be applied in order
          overlays = [ overlay ];
        };

    in rec {
      # For our outputs, we'll return the modified Yarn package from our overridden nixpkgs.
      packages = {
        yarn = pkgs.yarn;
      };
    }
  );
}

[ahobson]

With pinning to a commit, I think this is now possible to do

https://www.jetpack.io/devbox/docs/guides/pinning_packages/#manually-pinning-a-nixpkg-commit-for-a-package

e.g.

$ devbox version
0.8.5
$ devbox init
$ devbox add nodejs@20.8
# ...
$ devbox run node --version
v20.8.0
$ grep '#nodejs' devbox.lock
      "resolved": "github:NixOS/nixpkgs/c182df2e68bd97deb32c7e4765adfbbbcaf75b60#nodejs_20",
$ devbox add github:NixOS/nixpkgs/c182df2e68bd97deb32c7e4765adfbbbcaf75b60#nodejs_20.pkgs.yarn
# ...
$ devbox run node --version
v20.8.0
$ devbox run yarn node --version
yarn node v1.22.19
v20.8.0
✨  Done in 0.03s.

A bit cumbersome, but possible!

[AnhQuanTrl]

This is a serious bug that need to be addressed. It defeat the point of being an isolated dev environment.

[beetleman]

Similar problem I have with clojure and jvm runtime. Flakes are not the best solution for it because I do not want write nix

[jasononeil]

We've set up a devbox plugin that uses Corepack to set the Node and Yarn versions correctly based on the packageManager field in package.json.

The plugin is very simple: https://github.com/cultureamp/devbox-extras/blob/main/plugins/corepack/plugin.json

We're going to start trying this internally at Culture Amp, but it's hosted on a public repo so it should work for others if you want to try.

README here: https://github.com/cultureamp/devbox-extras/tree/main/plugins/corepack

I'm also open to contributing it as an official plugin if this approach is one Jetpack is happy with

[Lagoja]

I tested this quickly as a builtin plugin and it seems to work really well!

It doesn't seem to interfere with any other settings a user might want for Nodejs, so it's probably safe to enable it by default.

I'll push a PR with the plugin today

[Lagoja]

@jasononeil Thanks for sharing the plugin! I've been testing it on main and it seems to be working pretty well with the example projects.

If you're ok with contributing it, I'd like to merge it in as a builtin so we can have it work for all Node projects by default. I think this would fix a pretty huge painpoint.

[jasononeil]

Sounds good, thank you!