upgrade broke node_modules names

[shmish111]

we used to have a project working with an older version of yarn2nix, what we did was use the yarn.nix offline_cache however with recent versions the names of everything have changed e.g. webpack-cli-3.2.3.tgz has become webpack_cli___webpack_cli_3.2.3.tgz, this means that the offline cache doesn’t have anything I’m looking for since the names are all wrong. How should I be populating node_modules with the current version of yarn2nix?

[shmish111]

ok, I'm confused here. With master branch of yarn2nix I get the following

builder for '/nix/store/lq4i8yndrnp05yzngskavxlblbsjac61-meadow-client.drv' failed with exit code 1; last 10 log lines:
  success Set "yarn-offline-mirror" to "/nix/store/2rx8k64zwsy3ghigjrxlvfw7y0226ypn-offline".
  Done in 0.04s.
  yarn config v1.15.2
  success Set "yarn-offline-mirror-pruning" to "true".
  Done in 0.04s.
  yarn install v1.15.2
  [1/4] Resolving packages...
  [2/4] Fetching packages...
  error Can't make a request in offline mode ("https://registry.yarnpkg.com/spago/-/spago-0.7.7.tgz")
  info Visit https://yarnpkg.com/en/docs/cli/install for documentation about this command.

but

[root@nixops:~/yarn2nix]# ls -l /nix/store/2rx8k64zwsy3ghigjrxlvfw7y0226ypn-offline | grep spago
lrwxrwxrwx 2 root root  67 Jan  1  1970 spago___spago_0.7.7.tgz -> /nix/store/33xq80ckafa75gxbmnncx4b50wa27z0q-spago___spago_0.7.7.tgz

So yarn is looking for spago-0.7.7.tgz but the cache contains spago___spago_0.7.7.tgz If I now apply the following patch

diff --git a/lib/generateNix.js b/lib/generateNix.js
index 706364c..e4c0bc2 100644
--- a/lib/generateNix.js
+++ b/lib/generateNix.js
@@ -1,6 +1,6 @@
 const R = require('ramda')

-const urlToName = require('./urlToName')
+const path = require('path')

 // fetchgit transforms
 //
@@ -61,7 +61,7 @@ function fetchLockedDep(pkg) {

   const [url, sha1OrRev] = resolved.split('#')

-  const fileName = urlToName(url)
+  const fileName = path.basename(url)

   if (url.startsWith('git+')) {
     const rev = sha1OrRev

Then I get

builder for '/nix/store/f3kkmfz8g8v10n864abjnsg83fz1lk2q-meadow-modules-1.0.0.drv' failed with exit code 1; last 10 log lines:
  configuring
  building
  yarn config v1.15.2
  success Set "yarn-offline-mirror" to "/nix/store/1nz6ja10livki10anrl9hjmgzrdz41ag-offline".
  Done in 0.04s.
  yarn install v1.15.2
  [1/4] Resolving packages...
  [2/4] Fetching packages...
  error Can't make a request in offline mode ("https://registry.yarnpkg.com/spago___spago_0.7.7.tgz")
  info Visit https://yarnpkg.com/en/docs/cli/install for documentation about this command.

and

[root@nixops:~/yarn2nix]# ls -l /nix/store/1nz6ja10livki10anrl9hjmgzrdz41ag-offline | grep spago
lrwxrwxrwx 2 root root 59 Jan  1  1970 spago-0.7.7.tgz -> /nix/store/xf01agaiig34sgjby5a62kcga3nhqf0m-spago-0.7.7.tgz

So now spago-0.7.7.tgz exists in the cache but yarn is looking for spago___spago_0.7.7.tgz!

So everything is backwards somehow

[shmish111]

OK, so sorry for the noise, I had to look a bit deeper into mkYarnPackage and saw that everything is done in the configurePhase which I was overriding. I moved my stuff into the buildPhase and apart from a few paths needing juggling around it all worked. I will close this but if anyone comes across this in the future feel free to contact me as I might be able to help.

[shmish111]

I'm reopening this because I tried upgrading again recently and came up against the same issue. The offline cache has very weird entries like _webassemblyjs_ast___ast_1.8.5.tgz and events___events_3.0.0.tgz and I don't understand how yarn can possibly understand those.

The reason I tried upgrading is because one url is handled incorrectly, the following

"libxmljs@github:znerol/libxmljs#xmlwriter-0.19.5":
  version "0.19.5"
  resolved "https://codeload.github.com/znerol/libxmljs/tar.gz/6c01ab19552419873f62c37a8e2428641eee0498#aceb8427df4f7b46b33a6c04217e794bb0003098"
  dependencies:
    bindings "~1.3.0"
    nan "~2.10.0"

produces

    {
      name = "6c01ab19552419873f62c37a8e2428641eee0498";
      path = fetchurl {
        name = "6c01ab19552419873f62c37a8e2428641eee0498";
        url  = "https://codeload.github.com/znerol/libxmljs/tar.gz/6c01ab19552419873f62c37a8e2428641eee0498";
        sha1 = "aceb8427df4f7b46b33a6c04217e794bb0003098";
      };
    }

Notice the name field is wrong.

So at the moment I'm kind of stuck