[rush] `rush deploy` fails if it encounters a symbolic link in `node_modules`

salieri commented 3 years ago

Summary

Some packages, such as Puppeteer, automatically create symbolic links inside node_modules. rush deploy seems to fail when it encounters such a symlink.

Repro steps

Example at: https://github.com/salieri/rush-issue-with-symlink-and-deploy

This may be MacOS only issue, as I didn't notice it on a Linux VM.

Clone the repo
rush update
rush deploy --overwrite -p test

Expected result: Deploy command should complete successfully

Actual result:

% rush --debug deploy --overwrite -p test

Rush Multi-Project Build Tool 5.50.0 - https://rushjs.io
Node.js version is 14.17.0 (LTS)

Starting "rush deploy"

Loading deployment scenario: /path/to/common/config/rush/deploy.json
Deploying to target folder:  /path/to/common/deploy
Main project for deployment: test

Deleting target folder contents because "--overwrite" was specified...

Analyzing project: test

Copying folders...

ERROR: Internal Error: The path ends prematurely at: /path/to/common/temp/node_modules/.pnpm/puppeteer@10.1.0/node_modules/puppeteer/.local-chromium/mac-884014/chrome-mac/Chromium.app/Contents/Frameworks/Chromium
Framework.framework/Versions/Current/Chromium Framework
You have encountered a software defect. Please consider reporting the issue to the maintainers of this application.

Error: Internal Error: The path ends prematurely at: /path/to/common/temp/node_modules/.pnpm/puppeteer@10.1.0/node_modules/puppeteer/.local-chromium/mac-884014/chrome-mac/Chromium.app/Contents/Frameworks/Chromium Framework.framework/Versions/Current/Chromium Framework

You have encountered a software defect. Please consider reporting the issue to the maintainers of this application.
    at SymlinkAnalyzer.analyzePath (/Users/<user>/.rush/node-v14.17.0/rush-5.50.0/node_modules/@microsoft/rush-lib/lib/logic/deploy/SymlinkAnalyzer.js:105:23)
    at SymlinkAnalyzer.analyzePath (/Users/<user>/.rush/node-v14.17.0/rush-5.50.0/node_modules/@microsoft/rush-lib/lib/logic/deploy/SymlinkAnalyzer.js:77:45)
    at Object.filter (/Users/<user>/.rush/node-v14.17.0/rush-5.50.0/node_modules/@microsoft/rush-lib/lib/logic/deploy/DeployManager.js:299:53)
    at startCopy (/Users/<user>/.rush/node-v14.17.0/rush-5.50.0/node_modules/fs-extra/lib/copy-sync/copy-sync.js:35:28)
    at copyDirItem (/Users/<user>/.rush/node-v14.17.0/rush-5.50.0/node_modules/fs-extra/lib/copy-sync/copy-sync.js:118:10)
    at /Users/<user>/.rush/node-v14.17.0/rush-5.50.0/node_modules/fs-extra/lib/copy-sync/copy-sync.js:111:39
    at Array.forEach (<anonymous>)
    at copyDir (/Users/<user>/.rush/node-v14.17.0/rush-5.50.0/node_modules/fs-extra/lib/copy-sync/copy-sync.js:111:23)
    at mkDirAndCopy (/Users/<user>/.rush/node-v14.17.0/rush-5.50.0/node_modules/fs-extra/lib/copy-sync/copy-sync.js:106:3)
    at onDir (/Users/<user>/.rush/node-v14.17.0/rush-5.50.0/node_modules/fs-extra/lib/copy-sync/copy-sync.js:97:37)

Details

The issue is caused by SymlinkAnalyzer.ts expecting nothing but 'real' (non-symlink) directories in the path.

Standard questions

Please answer these questions to help us investigate your issue more quickly:

Question	Answer
`@microsoft/rush` globally installed version?	5.47.0
`rushVersion` from rush.json?	5.50.0
`useWorkspaces` from rush.json?	true
Operating system?	MacOS
Would you consider contributing a PR?	Maybe
Node.js version (`node -v`)?	14.17.0

salieri commented 3 years ago

Workaround if you don't care about having chromium with Puppeteer when you prepare your deployment package: PUPPETEER_SKIP_CHROMIUM_DOWNLOAD=1 rush update --full --purge --recheck

jeanlescure commented 2 years ago

Being affected by this, nothing to do with puppeteer, our project just underwent a very successful refactor (all tests passing in multiple packages/apps) that requires the usage of symlinks. But now our CI fails when deploying, in the Copying folders... stage.

Is there no way to ignore or bypass this check?

bryanmig commented 2 years ago

I have recently started seeing this same behavior in my Rush repo as well. I'm not sure what changed in our repo or environment but I've traced this back to a change in node-gyp (see https://github.com/nodejs/node-gyp/issues/2713) which intentionally creates a symlink to python on the host machine.

It looks like there is a fix that has not yet landed (https://github.com/nodejs/node-gyp/pull/2721) but my solution to this particular problem is simple. I've added a postRushInstall hook that finds and removes these dangling symlinks:

    "postRushInstall": [
      // This is needed to remove node_gyp_bins which contains symlinks outside the root
      // @see https://github.com/nodejs/node-gyp/issues/2713
      // @see https://github.com/nodejs/node-gyp/pull/2721
      "find ./common -name node_gyp_bins -type d -exec rm -r '{}' \\;"
    ],

microsoftly commented 1 year ago

I just encountered this too. My guess is that there's a bug with the folder copy where it doesn't properly handle paths with spaces in them

microsoft / rushstack