yarnpkg / berry

📦🐈 Active development trunk for Yarn ⚒
https://yarnpkg.com
BSD 2-Clause "Simplified" License
7.34k stars 1.1k forks source link

[Bug?]: Virtualized packages don't get their `link:`s virtualized #3819

Open raon0211 opened 2 years ago

raon0211 commented 2 years ago

Self-service

Describe the bug

We have many internal libraries linked through yarn workspaces. Most of them get virtualized properly, and their peer dependencies are correctly resolved.

However, some packages with link:s in their dependencies don't get properly virtualized. Problems occur when we refer to the the internal files of the library using the link: protocol. Even if the package itself is virtualized, for the dependency with link:, yarn uses the real path, not the virtualized path. Therefore, their peer dependencies get broken.

I think when yarn virtualizes packages, it has to virtualize the link: dependencies of the packages too.

To reproduce

const fs = require('fs');

await packageJson({
  workspaces: ['packages/*'],
});

/* There are two internal packages. */
await fs.promises.mkdir('./packages');
await fs.promises.mkdir('./packages/package-1');
await fs.promises.mkdir('./packages/package-2');

/* 
 * One is a library, with a dev dependency of react@^16.
 * It accepts any version of react as a peer dependency.
 **/
await packageJson({
  name: 'package-1',
  private: true,
  main: './src/index.js',
  dependencies: {
    src: 'link:./src',
  },
  devDependencies: {
    react: '^16',
  },
  peerDependencies: {
    react: '*',
  },
}, { cwd: './packages/package-1' });

/**
 * This library exposes a function returning the JavaScript file path of
 * which react it refers to.
 */
await fs.promises.mkdir('./packages/package-1/src');
await fs.promises.mkdir('./packages/package-1/src/utils');

await fs.promises.writeFile(
  './packages/package-1/src/utils/util-1.js',
  `module.exports = function getReactMainJSPath() {
    return require.resolve('react');
  };`
);

/**
 * Since dependencies with `link:` do not virtualize,
 * this line makes `src/utils/util-1.js` to be resolved in the real path, not the virtualized path.
 * Hence the peer dependencies get broken.
 **/
await fs.promises.writeFile(
  './packages/package-1/src/index.js',
  `module.exports = require('src/utils/util-1.js')`
);

/**
 * package-2 uses package-1 and provides react@^17 as a peer dependency of package-1.
 */
await packageJson({
  name: 'package-2',
  private: true,
  dependencies: {
    'package-1': 'workspace:^',
    react: '^17',
  },
}, { cwd: './packages/package-2' });

await yarn('install');

/** 
  * The expected return value is the path of React 17. 
  * However, the path to react 16 is returned.
  **/
await expect(node(`{
  const getReactMainJSPath = require('package-1'); 
  return getReactMainJSPath();
}`, { cwd: './packages/package-2' })).resolves.toContain('17');

Environment

System:
    OS: macOS 11.5.2
    CPU: (8) arm64 Apple M1
  Binaries:
    Node: 16.13.0 - /private/var/folders/95/2tt_grkx6ms0m00895kdl1x14mbb99/T/xfs-88197035/node
    Yarn: 3.1.0 - /private/var/folders/95/2tt_grkx6ms0m00895kdl1x14mbb99/T/xfs-88197035/yarn
    npm: 8.1.0 - ~/.nvm/versions/node/v16.13.0/bin/npm

Additional context

Simplified version of our .pnp.cjs

["@raon0211/our-internal-library", [
  ["virtual:34614177e0bce1b1e537468e6ed4ea6e125e8c4277168145902ba62a9129a0ea2b712515d0022793869bf95f550625e11c1843d166ddaa6b1c9371a843d97897#workspace:libraries/our-internal-library", {
    /* Here our package is correctly virtualized */
    "packageLocation": "./.yarn/__virtual__/@raon0211-our-internal-library-virtual-af81e9db79/1/libraries/our-internal-library/",
    "packageDependencies": [
        /* We refer our utils through this link: dependency */
        ["utils", "link:./src/utils::locator=%40raon0211%2Four-internal-library%40workspace%3Alibraries%2Four-internal-library"],
    ]
  }]
],
["utils", [
  ["link:./src/utils::locator=%40raon0211%2Four-internal-library%40workspace%3Alibraries%2Four-internal-library", {
    /* Here the package is not virtualized, breaking the peer dependencies */
    "packageLocation": "./libraries/our-internal-library/src/utils/",
    "packageDependencies": [
      ["utils", "link:./src/utils::locator=%40raon0211%2Four-internal-library%40workspace%3Alibraries%2Four-internal-library"]
    ],
    "linkType": "SOFT",
    "discardFromLookup": true
}]
raon0211 commented 2 years ago

I'm thinking of implementing a fix somewhere here.

yarnbot commented 2 years ago

This issue reproduces on master:

Error: expect(received).resolves.toContain(expected) // indexOf

Expected substring: "17"
Received string:    "/tmp/tmp-19kplOh1rrrw7q/.yarn/cache/react-npm-16.14.0-932446ec69-8484f3ecb1.zip/node_modules/react/index.js"
    at Object.toContain (/github/workspace/.yarn/cache/expect-npm-24.8.0-8c7640c562-0c0da74930.zip/node_modules/expect/build/index.js:202:20)
    at module.exports (evalmachine.<anonymous>:79:48)
    at processTicksAndRejections (node:internal/process/task_queues:96:5)
    at async /github/workspace/.yarn/cache/@arcanis-sherlock-npm-2.0.3-558f52b79f-ccfa417b92.zip/node_modules/@arcanis/sherlock/lib/executeRepro.js:57:13
    at async executeInTempDirectory (/github/workspace/.yarn/cache/@arcanis-sherlock-npm-2.0.3-558f52b79f-ccfa417b92.zip/node_modules/@arcanis/sherlock/lib/executeRepro.js:18:16)
    at async executeRepro (/github/workspace/.yarn/cache/@arcanis-sherlock-npm-2.0.3-558f52b79f-ccfa417b92.zip/node_modules/@arcanis/sherlock/lib/executeRepro.js:25:12)
    at async ExecCommand.execute (/github/workspace/.yarn/cache/@arcanis-sherlock-npm-2.0.3-558f52b79f-ccfa417b92.zip/node_modules/@arcanis/sherlock/lib/commands/exec.js:26:38)
    at async ExecCommand.validateAndExecute (/github/workspace/.yarn/cache/clipanion-npm-2.0.0-rc.16-b9444aaf89-91cf93ba72.zip/node_modules/clipanion/lib/advanced/Command.js:161:26)
    at async Cli.run (/github/workspace/.yarn/cache/clipanion-npm-2.0.0-rc.16-b9444aaf89-91cf93ba72.zip/node_modules/clipanion/lib/advanced/Cli.js:74:24)
    at async Cli.runExit (/github/workspace/.yarn/cache/clipanion-npm-2.0.0-rc.16-b9444aaf89-91cf93ba72.zip/node_modules/clipanion/lib/advanced/Cli.js:83:28)