Inside docker build Nx can't find project, but finds it on re-run of same docker build command

charlesritchea commented 7 months ago

Current Behavior

I have a Lerna/Nx monorepo with yarn workspaces. I'm not using Nx for anything other than Lerna runner at the moment Lerna and Nx are the latest, but this happened with earlier major version as well

We have an intermittent docker build failure that seems most likely to occur after merging latest from main.

Why I think this is a bug is because re-running the exact same docker build command always succeeds after this error. it makes CI a PITA because I have to manually re-run Github Actions just because of this.

Could we at least get better logging

Expected Behavior

Nx should not error here. @bar/foo-project absolutely does exist where it should and I've even run yarn install within the same docker RUN command to make sure symlinks are generated just in case even though I already ran yarn install in a previous dependencies layer (I've been trying lots of solutions to this problem) I've also run tree to make sure there wasn't an issue with docker COPY/ADD

GitHub Repo

No response

Steps to Reproduce

docker build yarn workspaces + lerna typescript repo. Can't send you my code sorry. I know this is not helpful, so maybe better logging to figure out why it think there is no project that is clearly there

Nx Report

Node   : 20.12.2
OS     : linux-x64
yarn   : 1.22.21

nx (global)    : 18.3.2
nx             : 18.2.4
lerna          : 8.1.2
@nx/js         : 18.2.4
@nx/workspace  : 18.2.4
@nx/devkit     : 18.2.4
@nrwl/tao      : 18.2.4
@nx/vite       : 18.2.4
typescript     : 4.9.5
---------------------------------------
Community plugins:
@analogjs/platform            : 1.2.0
@analogjs/vite-plugin-angular : 1.2.0

Failure Logs

NX   Could not find project "@bar/foo-project"
    Error: Could not find project "@bar/foo-project"
        at validateProject (/srv/node_modules/nx/src/command-line/run/run.js:24:15)
        at runExecutorInternal (/srv/node_modules/nx/src/command-line/run/run.js:92:5)
        at /srv/node_modules/nx/src/command-line/run/run.js:179:50
        at handleErrors (/srv/node_modules/nx/src/utils/params.js:9:22)
        at run (/srv/node_modules/nx/src/command-line/run/run.js:177:38)
        at process.<anonymous> (/srv/node_modules/nx/bin/run-executor.js:59:48)
        at process.emit (node:events:518:28)
        at emit (node:internal/child_process:951:14)
        at process.processTicksAndRejections (node:internal/process/task_queues:83:21)

Package Manager Version

1.22.21

Operating System

[ ] macOS
[X] Linux
[ ] Windows
[ ] Other (Please specify)

Additional Information

base image is node:20-alpine (maybe some weird musl issue?)

charlesritchea commented 7 months ago

It's possible this is a Lerna bug becaue we used to see similar behaviour when we used lerna bootstrap. Instead of this error bootstrap would silently skip some projects for no known reason, and would succeed on re-run.

charlesritchea commented 7 months ago

Are there any known paths that should be put into RUN mount cache or in .dockerignore related to Nx?

github-actions[bot] commented 1 month ago

This issue has been automatically marked as stale because it hasn't had any activity for 6 months. Many things may have changed within this time. The issue may have already been fixed or it may not be relevant anymore. If at this point, this is still an issue, please respond with updated information. It will be closed in 21 days if no further activity occurs. Thanks for being a part of the Nx community! 🙏

nrwl / nx