hedgedoc does not build on master

[yu-re-ka]

as mentioned in #175972

(in node_modules/sqlite3)

npm run install --build-from-source --nodedir=/nix/store/nbayk94b9w60mgdq9640241jr24hrdw4-nodejs-16.15.1/include/node

exits with code 243

responsible commit is e6188b65767b0cbce493dc9724e55ca005284331

[fatho]

I encountered the same symptom while building vscode-extensions.vadimcn.vscode-lldb. It too failed with exit code 243 on an npm run webpack ... line.

Thanks to you tracking it down to the nodejs update, I found that the biggest change was an upgrade of the included npm version from 8.5-something to 8.10 or so.

And for npm 8.6, there seems to be this matching issue: https://github.com/npm/cli/issues/4769

Apparently, it now fails when the home-directory does not exist. And inside sandboxed nix builds, there is no home directory.

I got the vscode-lldb build to work with a small hack, but that's obviously not a general solution:

      codelldb = pkgs.vscode-extensions.vadimcn.vscode-lldb.overrideAttrs (old: {
        preConfigure = ''
          mkdir /tmp/nodejs-home
          export HOME=/tmp/nodejs-home
        '';
      });

[phaer]

prepareAndInvokeNPM which is used by buildNodeDependencies just sets export HOME=$TMPDIR which seems to be /build in practice.

Adding export HOME=$TMPDIR to the beginning of buildPhase fixes both hedgedoc as well as jellyfin-web (mentioned in related issue above) for me.

As this issue seems to concern pretty much all node packages which call npm manually during their buildPhase, it might make sense to set HOME for all of them. Not sure yet whats the best way to do so.

[marius851000]

This issue also affect the peertube service. It set it's home directory to the peertube derivation, and so, it doesn't start. @mohe2015 (or another peertube maintainer), you may be interested in this issue (not sure it's a good idea to ping the whole peertube maintainers maintainer team...)

[BBBSnowball]

This works for me as a temporary fix:

  nixpkgs.overlays = [
    (self: super: {
       hedgedoc = super.hedgedoc.overrideAttrs (old: {
         # see https://github.com/NixOS/nixpkgs/issues/176127#issuecomment-1146782555
         preBuild = ''
           export HOME=$TMPDIR
         '';
       });
    })
  ];

Should we commit a fix for hedgedoc while we are thinking about more general solutions?

The responsible commit is a patch update of nodejs but this seems to do six minor updates for npm (see this and this).

I think it might be interesting to know why the newer npm insist on having a home. strace shows several statx and one openat on paths in $HOME but none of this is fatal. Then, it tries mkdir and bails out. The stacktrace has libuv and libc so I assume that the call is from Javascript and I don't know an easy way to find out more from the strace.

If it can access its home, it will create a .npm directory with _update-notifier-last-checked, _cacache and _logs. This first one should be fine. The location of the others can be configured by cache and logs-dir. The important one seems to be cache (the error code is not 243 but I think this is because the mkdir will see EROFS rather than ENOENT here):

$ export HOME=/nix/store/homeless
$ npm help ; echo $?
254
$ npm help --logs-dir=y ; echo $?
254
$ npm help --cache=x ; echo $?
npm <command>
...
$ find x
x
x/_logs
x/_logs/2022-06-06T21_08_26_555Z-debug-0.log
$ npm --cache=x2 run install --build-from-source --nodedir=/nix/store/nbayk94b9w60mgdq9640241jr24hrdw4-nodejs-16.15.1/include/node
... (lots of output, does not abort immediately)

As this also breaks the npm help command, we should maybe report this upstream rather than work around it? I think this might be on purpose but I would expect a useful error message in that case. What do you think?

Regarding a fix in nixpkgs: We can set $npm_config_cache instead of $HOME (e.g. export npm_config_cache=$TMPDIR/.npm). This should do the same thing as passing --cache but we can apply it in a hook script. As this should only be used by npm (in contrast to $HOME), I think we could apply it to all builds that are using npm. Here is the documentation on using env vars for config and it seems to work as expected:

$ npm_config_cache=x3 npm help ; echo $?
npm <command>
...
0

[mohe2015]

This issue also affect the peertube service. It set it's home directory to the peertube derivation, and so, it doesn't start. @mohe2015 (or another peertube maintainer), you may be interested in this issue (not sure it's a good idea to ping the whole peertube maintainers maintainer team...)

Currently not using NixOS so pinging @Izorkin

[pinpox]

This works for me as a temporary fix:

Tried this fix on latest unstable, but it still fails to build. This is what I got from nixos-rebuild

building '/nix/store/p2q1h6bs2gapsqcfbjqrrby75kz5j3jk-unit-systemd-udev-settle.service.drv'...
building '/nix/store/8iw002nmpqv682pv135ik6pdcirsifqh-unit-systemd-udevd.service.drv'...
building '/nix/store/3nkj3m79vlhsll64dq870lsgfwnkkhxs-unit-systemd-update-utmp.service.drv'...
93% after chunk asset optimization SourceMapDevToolPlugin index.cc145078bb69406a9fc3.js attach SourceMap/nix/store/ypszvxqskanxpxplcda0pfa6kmj2yjy0-stdenv-linux/setup: line 1387:   354 Killed                  npm run builds/ui/toolbar.cssmm2m2K2KmKs=emojify!emojify.js flowchart.js js-sequence-diagrams expose-loader?exposes=Reveal!reveal.js expose-loader?exposes=RevealMarkdown!reveal-markdown /build/source/deps/HedgeDoc/public/js/slide.jssimplescrollbars.css /build/source/deps/HedgeDoc/node_modules/codemirror/addon/search/matchesonscrollbar.css /build/source/deps/HedgeDoc/node_modules/codemirror/theme/monokai.css /build/source/deps/HedgeDoc/node_modules/codemirror/theme/one-dark.css /build/source/deps/HedgeDoc/node_modules/codemirror/mode/tiddlywiki/tiddlywiki.css /build/source/deps/HedgeDoc/node_modules/codemirror/mode/mediawiki/mediawiki.css /build/source/deps/HedgeDoc/node_modules/spin.js/spin.css /build/source/deps/HedgeDoc/public/css/github-extract.css /build/source/deps/HedgeDoc/public/vendor/showup/showup.css /build/source/deps/HedgeDoc/public/css/mermaid.css /build/source/deps/HedgeDoc/public/css/markdown.css /build/source/deps/HedgeDoc/public/css/slide-preview.css
building '/nix/store/yjpvcx8lrqzdr9mcpgd206b3kiigr82k-unit-systemd-user-sessions.service.drv'...
error: builder for '/nix/store/fas34457n3m77vk46c334plqf8ipmbs3-HedgeDoc-1.9.0.drv' failed with exit code 137
error: 1 dependencies of derivation '/nix/store/ipn37hrbvis2qainsl9ckcywc4jvbzgs-unit-hedgedoc.service.drv' failed to build
error: 1 dependencies of derivation '/nix/store/xhd877x733msg5fjhvmjvvbh2c8zhy1l-system-units.drv' failed to build
building '/nix/store/rj4zda80washrnnqfzpv6adzka1nbymv-user-units.drv'...
building '/nix/store/3fxxydg5fq1xjp7qnihbq9q730nr6xrd-useradd.drv'...
error: 1 dependencies of derivation '/nix/store/qcix9mr1yji5nms6p222bphd6w0171hn-etc.drv' failed to build
error: 1 dependencies of derivation '/nix/store/klz8jgki9pms30m52ixrxgvvckv58y3i-nixos-system-kfbox-22.11.20220608.e0169d7.drv' failed to build

[NickHu]

@pinpox To me, it looks like your machine ran out of memory while building so it got killed

[Izorkin]

Currently not using NixOS so pinging @Izorkin

Fix work PeerTube with nodejs 16.15.1

diff --git a/nixos/modules/services/web-apps/peertube.nix b/nixos/modules/services/web-apps/peertube.nix
index e6b6aa273e7..40838e5807a 100644
--- a/nixos/modules/services/web-apps/peertube.nix
+++ b/nixos/modules/services/web-apps/peertube.nix
@@ -409,6 +409,7 @@ in {
           password: '$(cat ${cfg.smtp.passwordFile})'
         ''}
         EOF
+        mkdir /tmp/.npm
         ln -sf ${cfg.package}/config/default.yaml /var/lib/peertube/config/default.yaml
         ln -sf ${configFile} /var/lib/peertube/config/production.json
         npm start
@@ -433,7 +434,7 @@ in {
         RestrictAddressFamilies = [ "AF_UNIX" "AF_INET" "AF_INET6" "AF_NETLINK" ];
         MemoryDenyWriteExecute = false;
         # System Call Filtering
-        SystemCallFilter = [ ("~" + lib.concatStringsSep " " systemCallsList) "pipe" "pipe2" ];
+        SystemCallFilter = [ ("~" + lib.concatStringsSep " " systemCallsList) "@chown" "pipe" "pipe2" ];
       } // cfgService;
     };
diff --git a/pkgs/servers/peertube/default.nix b/pkgs/servers/peertube/default.nix
index 558c21c6cd1..8c107047af2 100644
--- a/pkgs/servers/peertube/default.nix
+++ b/pkgs/servers/peertube/default.nix
@@ -97,6 +97,7 @@ in stdenv.mkDerivation rec {
     mkdir $out/client
     mv ~/client/{dist,node_modules,package.json,yarn.lock} $out/client
     mv ~/{config,scripts,support,CREDITS.md,FAQ.md,LICENSE,README.md,package.json,tsconfig.json,yarn.lock} $out
+    ln -s /tmp/.npm $out/.npm
   '';

   passthru.tests.peertube = nixosTests.peertube;

[Izorkin]

After that PR npm started to give an error of creating temporary files - https://github.com/npm/cli/pull/4594 Issue - https://github.com/npm/cli/issues/4996