[Bug?]: `yarn rebuild <module>` still rebuilds all other modules that haven't previously been built

yarnpkg / berry

📦🐈 Active development trunk for Yarn ⚒

BSD 2-Clause "Simplified" License

7.43k stars 1.11k forks source link

Self-service

[x] I'd be willing to implement a fix

Describe the bug

If I have two dependencies with post-install scripts, e.g. esbuild and xsd-schema-validator, running yarn rebuild esbuild will cause the post-install script for xsd-schema-validator to also be run. Per the docs:

By default all packages will be rebuilt, but you can filter the list by specifying the names of the packages you want to clear from memory. Rebuild fsevents only :
yarn rebuild fsevents

Note that this only happens if xsd-schema-validator has not been built yet - if it's last build failed or no attempt has been made to build it previously.

The only reason I'm trying to build a select list of modules is that I'm working on a monorepo where builds are disabled for regular yarn install (on a developer's machine) but are needed in other environments e.g. in Dockerfiles. Additionally, the Dockerfiles only build certain modules that are needed in the eventual container. There's currently no feature that allows building certain modules in certain environments - a combination of enableScripts: false and dependenciesMeta only allows configuring a list of modules to build in all environments. Also, I'd prefer to not use dependenciesMeta for a deny-list as that is not very maintainable.

To reproduce

Playground

process.env.YARN_ENABLE_SCRIPTS = 'false';

const installPromise = packageJsonAndInstall({
dependencies: {
[`esbuild`]: `0.12.11`,
[`xsd-schema-validator`]: `0.7.0`,
},
});

await expect(installPromise)
.resolves.toContain(`xsd-schema-validator@npm:0.7.0 lists build scripts, but all build scripts have been disabled`);

process.env.YARN_ENABLE_SCRIPTS = 'true';

const rebuildPromise = yarn(`rebuild`, `esbuild`)

await expect(rebuildPromise)
.resolves.not.toContain(`xsd-schema-validator`);

Output:

Error: [2mexpect([22m[31mreceived[39m[2m).[22mresolves[2m.[22mnot[2m.[22mtoContain[2m()[22m

Received promise rejected instead of resolved
Rejected to value: [31m[Error: Command failed: /home/sandbox/.nvm/versions/node/v14.16.1/bin/node /sandbox/berry/scripts/actions/../run-yarn.js rebuild esbuild·[39m
[31m➤ YN0000: ┌ Resolution step[39m
[31m➤ YN0000: └ Completed[39m
[31m➤ YN0000: ┌ Fetch step[39m
[31m➤ YN0000: └ Completed[39m
[31m➤ YN0000: ┌ Link step[39m
[31m➤ YN0007: │ esbuild@npm:0.12.11 must be built because it never has been before or the last one failed[39m
[31m➤ YN0007: │ xsd-schema-validator@npm:0.7.0 must be built because it never has been before or the last one failed[39m
[31m➤ YN0000: │ xsd-schema-validator@npm:0.7.0 STDOUT [xsd-schema-validator] Compiling helper...[39m
[31m➤ YN0000: │ xsd-schema-validator@npm:0.7.0 STDERR Error: Java SDK required at JAVA_HOME or in path to compile validation helper[39m
[31m➤ YN0000: │ xsd-schema-validator@npm:0.7.0 STDERR     at /tmp/tmp-61LAIR84tWXcQs/.yarn/unplugged/xsd-schema-validator-npm-0.7.0-172763c240/node_modules/xsd-schema-validator/lib/validator.js:61:9[39m
[31m➤ YN0009: │ xsd-schema-validator@npm:0.7.0 couldn't be built successfully (exit code 1, logs can be found here: /tmp/xfs-aa2d3681/build.log)[39m
[31m➤ YN0000: └ Completed in 0s 290ms[39m
[31m➤ YN0000: Failed with errors in 0s 352ms[39m
[31m][39m
at expect (/sandbox/node_modules/expect/build/index.js:138:15)
at module.exports (evalmachine.<anonymous>:18:7)
at runMicrotasks (<anonymous>)
at processTicksAndRejections (internal/process/task_queues.js:93:5)
at async /sandbox/node_modules/@arcanis/sherlock/lib/executeRepro.js:56:13
at async executeInTempDirectory (/sandbox/node_modules/@arcanis/sherlock/lib/executeRepro.js:17:16)
at async executeRepro (/sandbox/node_modules/@arcanis/sherlock/lib/executeRepro.js:24:12)
at async module.exports../pages/api/sherlock.js.__webpack_exports__.default (/sandbox/.next/server/static/development/pages/api/sherlock.js:140:29)
at async Object.apiResolver (/sandbox/node_modules/next/dist/next-server/server/api-utils.js:46:9)
at async DevServer.handleApiRequest (/sandbox/node_modules/next/dist/next-server/server/next-server.js:449:9)
at async Object.fn (/sandbox/node_modules/next/dist/next-server/server/next-server.js:371:37)
at async Router.execute (/sandbox/node_modules/next/dist/next-server/server/router.js:134:32)
at async DevServer.run (/sandbox/node_modules/next/dist/next-server/server/next-server.js:491:29)
at async DevServer.handleRequest (/sandbox/node_modules/next/dist/next-server/server/next-server.js:144:20)

Environment

System: OS: Linux 5.4 Ubuntu 20.04.2 LTS (Focal Fossa) CPU: (4) x64 Intel(R) Core(TM) i7-5600U CPU @ 2.60GHz Binaries: Node: 14.15.1 - /tmp/xfs-2343c956/node Yarn: 3.0.0-rc.12.git.20210723.hash-b2a97475 - /tmp/xfs-2343c956/yarn npm: 6.14.8 - ~/.nvm/versions/node/v14.15.1/bin/npm

Additional context

No response

Error: expect(received).resolves.not.toContain() Received promise rejected instead of resolved Rejected to value: [Error: Command failed: /usr/bin/node /github/workspace/scripts/actions/../run-yarn.js rebuild esbuild ➤ YN0000: ┌ Resolution step ::group::Resolution step ::endgroup:: ➤ YN0000: └ Completed ➤ YN0000: ┌ Fetch step ::group::Fetch step ➤ YN0013: │ 4 packages were already cached ::endgroup:: ➤ YN0000: └ Completed ➤ YN0000: ┌ Link step ::group::Link step ➤ YN0007: │ esbuild@npm:0.12.11 must be built because it never has been before or the last one failed ➤ YN0007: │ xsd-schema-validator@npm:0.7.0 must be built because it never has been before or the last one failed ➤ YN0000: │ esbuild@npm:0.12.11 STDERR Trying to install "esbuild-linux-64" using npm ➤ YN0000: │ esbuild@npm:0.12.11 STDERR Failed to install "esbuild-linux-64" using npm: Command failed: npm install --loglevel=error --prefer-offline --no-audit --progress=false esbuild-linux-64@0.12.11 ➤ YN0000: │ esbuild@npm:0.12.11 STDERR /bin/sh: npm: not found ➤ YN0000: │ esbuild@npm:0.12.11 STDERR ➤ YN0000: │ esbuild@npm:0.12.11 STDERR Trying to download "https://registry.npmjs.org/esbuild-linux-64/-/esbuild-linux-64-0.12.11.tgz" ➤ YN0000: │ xsd-schema-validator@npm:0.7.0 STDOUT [xsd-schema-validator] Compiling helper... ➤ YN0000: │ xsd-schema-validator@npm:0.7.0 STDERR Error: Java SDK required at JAVA_HOME or in path to compile validation helper ➤ YN0000: │ xsd-schema-validator@npm:0.7.0 STDERR at /tmp/tmp-18Mu02QSKlOdMr/.yarn/unplugged/xsd-schema-validator-npm-0.7.0-172763c240/node_modules/xsd-schema-validator/lib/validator.js:61:9 ➤ YN0009: │ xsd-schema-validator@npm:0.7.0 couldn't be built successfully (exit code 1, logs can be found here: /tmp/xfs-0b7eabfd/build.log) ➤ YN0000: │ esbuild@npm:0.12.11 STDERR Install successful ::endgroup:: ➤ YN0000: └ Completed in 0s 654ms ➤ YN0000: Failed with errors in 0s 711ms ] at expect (/github/workspace/.yarn/cache/expect-npm-24.8.0-8c7640c562-0c0da74930.zip/node_modules/expect/build/index.js:138:15) at module.exports (evalmachine.<anonymous>:18:7) at processTicksAndRejections (internal/process/task_queues.js:95:5) at async /github/workspace/.yarn/cache/@arcanis-sherlock-npm-2.0.2-91650a2501-627bee24a7.zip/node_modules/@arcanis/sherlock/lib/executeRepro.js:56:13 at async executeInTempDirectory (/github/workspace/.yarn/cache/@arcanis-sherlock-npm-2.0.2-91650a2501-627bee24a7.zip/node_modules/@arcanis/sherlock/lib/executeRepro.js:17:16) at async Object.executeRepro (/github/workspace/.yarn/cache/@arcanis-sherlock-npm-2.0.2-91650a2501-627bee24a7.zip/node_modules/@arcanis/sherlock/lib/executeRepro.js:24:12) at async ExecCommand.execute (/github/workspace/.yarn/cache/@arcanis-sherlock-npm-2.0.2-91650a2501-627bee24a7.zip/node_modules/@arcanis/sherlock/lib/commands/exec.js:25:38) at async ExecCommand.validateAndExecute (/github/workspace/.yarn/cache/clipanion-npm-2.0.0-rc.16-b9444aaf89-91cf93ba72.zip/node_modules/clipanion/lib/advanced/Command.js:161:26) at async Cli.run (/github/workspace/.yarn/cache/clipanion-npm-2.0.0-rc.16-b9444aaf89-91cf93ba72.zip/node_modules/clipanion/lib/advanced/Cli.js:74:24) at async Cli.runExit (/github/workspace/.yarn/cache/clipanion-npm-2.0.0-rc.16-b9444aaf89-91cf93ba72.zip/node_modules/clipanion/lib/advanced/Cli.js:83:28)

Error: expect(received).resolves.not.toContain() Received promise rejected instead of resolved Rejected to value: [Error: Command failed: /usr/bin/node /github/workspace/scripts/actions/../run-yarn.js rebuild esbuild ➤ YN0000: ┌ Resolution step ::group::Resolution step ::endgroup:: ➤ YN0000: └ Completed ➤ YN0000: ┌ Fetch step ::group::Fetch step ➤ YN0013: │ 4 packages were already cached ::endgroup:: ➤ YN0000: └ Completed ➤ YN0000: ┌ Link step ::group::Link step ➤ YN0007: │ esbuild@npm:0.12.11 must be built because it never has been before or the last one failed ➤ YN0007: │ xsd-schema-validator@npm:0.7.0 must be built because it never has been before or the last one failed ➤ YN0000: │ esbuild@npm:0.12.11 STDERR Trying to install "esbuild-linux-64" using npm ➤ YN0000: │ esbuild@npm:0.12.11 STDERR Failed to install "esbuild-linux-64" using npm: Command failed: npm install --loglevel=error --prefer-offline --no-audit --progress=false esbuild-linux-64@0.12.11 ➤ YN0000: │ esbuild@npm:0.12.11 STDERR /bin/sh: npm: not found ➤ YN0000: │ esbuild@npm:0.12.11 STDERR ➤ YN0000: │ esbuild@npm:0.12.11 STDERR Trying to download "https://registry.npmjs.org/esbuild-linux-64/-/esbuild-linux-64-0.12.11.tgz" ➤ YN0000: │ xsd-schema-validator@npm:0.7.0 STDOUT [xsd-schema-validator] Compiling helper... ➤ YN0000: │ xsd-schema-validator@npm:0.7.0 STDERR Error: Java SDK required at JAVA_HOME or in path to compile validation helper ➤ YN0000: │ xsd-schema-validator@npm:0.7.0 STDERR at /tmp/tmp-18T22PIArV64tE/.yarn/unplugged/xsd-schema-validator-npm-0.7.0-172763c240/node_modules/xsd-schema-validator/lib/validator.js:61:9 ➤ YN0009: │ xsd-schema-validator@npm:0.7.0 couldn't be built successfully (exit code 1, logs can be found here: /tmp/xfs-24689234/build.log) ➤ YN0000: │ esbuild@npm:0.12.11 STDERR Install successful ::endgroup:: ➤ YN0000: └ Completed in 0s 631ms ➤ YN0000: Failed with errors in 0s 679ms ] at expect (/github/workspace/.yarn/cache/expect-npm-24.8.0-8c7640c562-0c0da74930.zip/node_modules/expect/build/index.js:138:15) at module.exports (evalmachine.<anonymous>:18:7) at processTicksAndRejections (internal/process/task_queues.js:95:5) at async /github/workspace/.yarn/cache/@arcanis-sherlock-npm-2.0.2-91650a2501-627bee24a7.zip/node_modules/@arcanis/sherlock/lib/executeRepro.js:56:13 at async executeInTempDirectory (/github/workspace/.yarn/cache/@arcanis-sherlock-npm-2.0.2-91650a2501-627bee24a7.zip/node_modules/@arcanis/sherlock/lib/executeRepro.js:17:16) at async Object.executeRepro (/github/workspace/.yarn/cache/@arcanis-sherlock-npm-2.0.2-91650a2501-627bee24a7.zip/node_modules/@arcanis/sherlock/lib/executeRepro.js:24:12) at async ExecCommand.execute (/github/workspace/.yarn/cache/@arcanis-sherlock-npm-2.0.2-91650a2501-627bee24a7.zip/node_modules/@arcanis/sherlock/lib/commands/exec.js:25:38) at async ExecCommand.validateAndExecute (/github/workspace/.yarn/cache/clipanion-npm-2.0.0-rc.16-b9444aaf89-91cf93ba72.zip/node_modules/clipanion/lib/advanced/Command.js:161:26) at async Cli.run (/github/workspace/.yarn/cache/clipanion-npm-2.0.0-rc.16-b9444aaf89-91cf93ba72.zip/node_modules/clipanion/lib/advanced/Cli.js:74:24) at async Cli.runExit (/github/workspace/.yarn/cache/clipanion-npm-2.0.0-rc.16-b9444aaf89-91cf93ba72.zip/node_modules/clipanion/lib/advanced/Cli.js:83:28)

Error: expect(received).resolves.not.toContain() Received promise rejected instead of resolved Rejected to value: [Error: Command failed: /usr/bin/node /github/workspace/scripts/actions/../run-yarn.js rebuild esbuild ➤ YN0000: ┌ Resolution step ::group::Resolution step ::endgroup:: ➤ YN0000: └ Completed ➤ YN0000: ┌ Fetch step ::group::Fetch step ➤ YN0013: │ 4 packages were already cached ::endgroup:: ➤ YN0000: └ Completed ➤ YN0000: ┌ Link step ::group::Link step ➤ YN0007: │ esbuild@npm:0.12.11 must be built because it never has been before or the last one failed ➤ YN0007: │ xsd-schema-validator@npm:0.7.0 must be built because it never has been before or the last one failed ➤ YN0000: │ esbuild@npm:0.12.11 STDERR Trying to install "esbuild-linux-64" using npm ➤ YN0000: │ esbuild@npm:0.12.11 STDERR Failed to install "esbuild-linux-64" using npm: Command failed: npm install --loglevel=error --prefer-offline --no-audit --progress=false esbuild-linux-64@0.12.11 ➤ YN0000: │ esbuild@npm:0.12.11 STDERR /bin/sh: npm: not found ➤ YN0000: │ esbuild@npm:0.12.11 STDERR ➤ YN0000: │ esbuild@npm:0.12.11 STDERR Trying to download "https://registry.npmjs.org/esbuild-linux-64/-/esbuild-linux-64-0.12.11.tgz" ➤ YN0000: │ xsd-schema-validator@npm:0.7.0 STDOUT [xsd-schema-validator] Compiling helper... ➤ YN0000: │ xsd-schema-validator@npm:0.7.0 STDERR Error: Java SDK required at JAVA_HOME or in path to compile validation helper ➤ YN0000: │ xsd-schema-validator@npm:0.7.0 STDERR at /tmp/tmp-190Ui5z2uv9wAU/.yarn/unplugged/xsd-schema-validator-npm-0.7.0-172763c240/node_modules/xsd-schema-validator/lib/validator.js:61:9 ➤ YN0009: │ xsd-schema-validator@npm:0.7.0 couldn't be built successfully (exit code 1, logs can be found here: /tmp/xfs-f89c2592/build.log) ➤ YN0000: │ esbuild@npm:0.12.11 STDERR Install successful ::endgroup:: ➤ YN0000: └ Completed in 0s 665ms ➤ YN0000: Failed with errors in 0s 718ms ] at expect (/github/workspace/.yarn/cache/expect-npm-24.8.0-8c7640c562-0c0da74930.zip/node_modules/expect/build/index.js:138:15) at module.exports (evalmachine.<anonymous>:18:7) at processTicksAndRejections (internal/process/task_queues.js:95:5) at async /github/workspace/.yarn/cache/@arcanis-sherlock-npm-2.0.2-91650a2501-627bee24a7.zip/node_modules/@arcanis/sherlock/lib/executeRepro.js:56:13 at async executeInTempDirectory (/github/workspace/.yarn/cache/@arcanis-sherlock-npm-2.0.2-91650a2501-627bee24a7.zip/node_modules/@arcanis/sherlock/lib/executeRepro.js:17:16) at async Object.executeRepro (/github/workspace/.yarn/cache/@arcanis-sherlock-npm-2.0.2-91650a2501-627bee24a7.zip/node_modules/@arcanis/sherlock/lib/executeRepro.js:24:12) at async ExecCommand.execute (/github/workspace/.yarn/cache/@arcanis-sherlock-npm-2.0.2-91650a2501-627bee24a7.zip/node_modules/@arcanis/sherlock/lib/commands/exec.js:25:38) at async ExecCommand.validateAndExecute (/github/workspace/.yarn/cache/clipanion-npm-2.0.0-rc.16-b9444aaf89-91cf93ba72.zip/node_modules/clipanion/lib/advanced/Command.js:161:26) at async Cli.run (/github/workspace/.yarn/cache/clipanion-npm-2.0.0-rc.16-b9444aaf89-91cf93ba72.zip/node_modules/clipanion/lib/advanced/Cli.js:74:24) at async Cli.runExit (/github/workspace/.yarn/cache/clipanion-npm-2.0.0-rc.16-b9444aaf89-91cf93ba72.zip/node_modules/clipanion/lib/advanced/Cli.js:83:28)

yarnpkg / berry