nrwl / nx

Smart Monorepos · Fast CI
https://nx.dev
MIT License
23.17k stars 2.3k forks source link

Building MFE apps in Dockerfile results in "Cannot read properties of undefined (reading 'indexOf')" in /app/node_modules/nx/src/project-graph/project-graph.js:228:27 #27078

Closed danielritter closed 3 weeks ago

danielritter commented 1 month ago

Current Behavior

When I want to build my MFE apps in my Dockerfile I get the following error in my TeamCity CI on RockyLinux:

19:46:42   #42 [frontend-angular-logout build  9/14] RUN NX_REJECT_UNKNOWN_LOCAL_CACHE=0 npx nx run frontend-angular-login:build:production
19:46:56   #42 13.94
19:46:56   #42 13.95  NX   Cannot read properties of undefined (reading 'indexOf')
19:46:56   #42 13.95
19:46:56   #42 13.95
19:46:56   #42 13.95 TypeError: Cannot read properties of undefined (reading 'indexOf')
19:46:56   #42 13.95     at createProjectGraphAndSourceMapsAsync (/app/node_modules/nx/src/project-graph/project-graph.js:228:27)
19:46:56   #42 13.95     at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
19:46:56   #42 13.95     at async createProjectGraphAsync (/app/node_modules/nx/src/project-graph/project-graph.js:198:39)
19:46:56   #42 13.95     at async Object.runOne (/app/node_modules/nx/src/command-line/run/run-one.js:24:26)
19:46:56   #42 13.95     at async /app/node_modules/nx/src/command-line/run/command-object.js:19:13
19:46:56   #42 13.95     at async handleErrors (/app/node_modules/nx/src/utils/params.js:22:24)
19:46:56   #42 13.95     at async Object.handler (/app/node_modules/nx/src/command-line/run/command-object.js:18:26)
19:46:56   #42 ERROR: process "/bin/sh -c NX_REJECT_UNKNOWN_LOCAL_CACHE=0 npx nx run frontend-angular-login:build:production" did not complete successfully: exit code: 1
19:46:56   failed to solve: process "/bin/sh -c NX_REJECT_UNKNOWN_LOCAL_CACHE=0 npx nx run frontend-angular-login:build:production" did not complete successfully: exit code: 1
19:46:56   ------
19:46:56    > [frontend-angular-development build  9/14] RUN NX_REJECT_UNKNOWN_LOCAL_CACHE=0 npx nx run frontend-angular-login:build:production:
19:46:56   13.95
19:46:56   13.95
19:46:56   13.95 TypeError: Cannot read properties of undefined (reading 'indexOf')
19:46:56   13.95     at createProjectGraphAndSourceMapsAsync (/app/node_modules/nx/src/project-graph/project-graph.js:228:27)
19:46:56   13.95     at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
19:46:56   13.95     at async createProjectGraphAsync (/app/node_modules/nx/src/project-graph/project-graph.js:198:39)
19:46:56   13.95     at async Object.runOne (/app/node_modules/nx/src/command-line/run/run-one.js:24:26)
19:46:56   13.95     at async /app/node_modules/nx/src/command-line/run/command-object.js:19:13
19:46:56   13.95     at async handleErrors (/app/node_modules/nx/src/utils/params.js:22:24)
19:46:56   13.95     at async Object.handler (/app/node_modules/nx/src/command-line/run/command-object.js:18:26)
19:46:56   ------
19:46:57   Process exited with code 17
19:46:16   Process exited with code 17 (Step: docker:compose:build (Command Line))
19:46:57   Step docker:compose:build (Command Line) failed

I use the node:lts-slim docker image to build my Dockerfile and run RUN NX_REJECT_UNKNOWN_LOCAL_CACHE=0 npx nx run frontend-angular-login:build:production.

Expected Behavior

If e.message is undefined in if (e.message.indexOf('inotify_add_watch') > -1) { on line /node_modules/nx/src/project-graph/project-graph.js:228:27 it results in the error above. I expect something like if (e.message && e.message.indexOf('inotify_add_watch') > -1) { to resolve the issue.

GitHub Repo

No response

Steps to Reproduce

  1. Building Dockerfile

Nx Report

Node   : 20.12.2
OS     : linux-x64
npm    : 10.8.2

nx                 : 19.5.2
@nx/js             : 19.5.2
@nx/jest           : 19.5.2
@nx/linter         : 19.5.2
@nx/eslint         : 19.5.2
@nx/workspace      : 19.5.2
@nx/angular        : 19.5.2
@nx/cypress        : 19.5.2
@nx/devkit         : 19.5.2
@nx/eslint-plugin  : 19.5.2
@nx/storybook      : 19.5.2
@nrwl/tao          : 19.5.2
@nx/web            : 19.5.2
@nx/webpack        : 19.5.2
typescript         : 5.4.5
---------------------------------------
Registered Plugins:
@nx/cypress/plugin
@nx/eslint/plugin
@nx/storybook/plugin
---------------------------------------
Community plugins:
@compodoc/compodoc : 1.1.25
@nativescript/nx   : 19.0.0
@storybook/angular : 8.1.11

Failure Logs

No response

Package Manager Version

npm

Operating System

Additional Information

Locally building Dockerfile on Windows it works fine.

Coly010 commented 1 month ago

Hey :) I'm going to need a minimal reproduction repo to investigate this further

kramar commented 1 month ago

We encounter the same issue when running a production build using nx in GitHub Actions (in Docker). Locally, on macOS, there are no errors, and the build completes successfully.

danielritter commented 1 month ago

Since it works successfully locally, it is difficult to provide a repository. But I have a multi stage Dockerfile with different new created MFE apps (seven of it), one storybook app and one compodoc app that looks something like:

docker-compose.yml

include:
    - tools/docker-compose/documentation-compodoc-frontend.docker-compose.yml
    - tools/docker-compose/frontend-angular-dashboard.docker-compose.yml
    - tools/docker-compose/frontend-angular-development.docker-compose.yml
    - tools/docker-compose/frontend-angular-login.docker-compose.yml
    - tools/docker-compose/frontend-angular-logout.docker-compose.yml
    - tools/docker-compose/frontend-angular-register.docker-compose.yml
    - tools/docker-compose/frontend-angular-shell.docker-compose.yml
    - tools/docker-compose/frontend-angular-ui-storybook.docker-compose.yml
    - tools/docker-compose/frontend-angular-user.docker-compose.yml

networks:
    frontend:

Dockerfile

FROM NEXUS_URL/node:lts-slim AS build
USER node
WORKDIR /app
COPY --chown=node:node . .
ARG NPM_TOKEN
ENV NPM_TOKEN=$NPM_TOKEN
ARG NPM_REGISTRY
ENV NPM_REGISTRY=$NPM_REGISTRY
ENV NX_DAEMON=true
ENV NX_VERBOSE_LOGGING=true
ARG APP_PORT
ENV APP_PORT=$APP_PORT
RUN npm config set "$NPM_REGISTRY:_authToken" $NPM_TOKEN
RUN npm ci
RUN npm run build:documentation:compodoc:frontend
RUN NX_REJECT_UNKNOWN_LOCAL_CACHE=0 npx nx run frontend-angular-dashboard:build:production
RUN NX_REJECT_UNKNOWN_LOCAL_CACHE=0 npx nx run frontend-angular-development:build:production
RUN NX_REJECT_UNKNOWN_LOCAL_CACHE=0 npx nx run frontend-angular-login:build:production
RUN NX_REJECT_UNKNOWN_LOCAL_CACHE=0 npx nx run frontend-angular-logout:build:production
RUN NX_REJECT_UNKNOWN_LOCAL_CACHE=0 npx nx run frontend-angular-register:build:production
RUN NX_REJECT_UNKNOWN_LOCAL_CACHE=0 npx nx run frontend-angular-shell:build:production
RUN NX_REJECT_UNKNOWN_LOCAL_CACHE=0 npx nx run frontend-angular-user:build:production
RUN NX_REJECT_UNKNOWN_LOCAL_CACHE=0 npx nx run frontend-angular-ui-storybook:build-storybook:production

FROM NEXUS_URL/nginx:stable-alpine-slim AS documentation-compodoc-frontend
WORKDIR /usr/share/nginx/html
COPY --from=build /app/tools/nginx/documentation-compodoc-frontend.conf /etc/nginx/nginx.conf
COPY --from=build /app/dist/libs/documentation/compodoc/frontend .
RUN rm -f 50x.html
ARG APP_PORT
ENV APP_PORT=$APP_PORT
EXPOSE $APP_PORT
CMD ["nginx", "-g", "daemon off;"]

FROM NEXUS_URL/nginx:stable-alpine-slim AS frontend-angular-dashboard
WORKDIR /usr/share/nginx/html
COPY --from=build /app/tools/nginx/frontend-angular-dashboard.conf /etc/nginx/nginx.conf
COPY --from=build /app/dist/apps/frontend/angular/dashboard/browser .
RUN rm -f 50x.html
RUN rm -f 3rdpartylicenses.txt
ARG APP_PORT
ENV APP_PORT=$APP_PORT
EXPOSE $APP_PORT
CMD ["nginx", "-g", "daemon off;"]

FROM NEXUS_URL/nginx:stable-alpine-slim AS frontend-angular-development
WORKDIR /usr/share/nginx/html
COPY --from=build /app/tools/nginx/frontend-angular-development.conf /etc/nginx/nginx.conf
COPY --from=build /app/dist/apps/frontend/angular/development/browser .
RUN rm -f 50x.html
RUN rm -f 3rdpartylicenses.txt
ARG APP_PORT
ENV APP_PORT=$APP_PORT
EXPOSE $APP_PORT
CMD ["nginx", "-g", "daemon off;"]

FROM NEXUS_URL/nginx:stable-alpine-slim AS frontend-angular-login
WORKDIR /usr/share/nginx/html
COPY --from=build /app/tools/nginx/frontend-angular-login.conf /etc/nginx/nginx.conf
COPY --from=build /app/dist/apps/frontend/angular/login/browser .
RUN rm -f 50x.html
RUN rm -f 3rdpartylicenses.txt
ARG APP_PORT
ENV APP_PORT=$APP_PORT
EXPOSE $APP_PORT
CMD ["nginx", "-g", "daemon off;"]

FROM NEXUS_URL/nginx:stable-alpine-slim AS frontend-angular-logout
WORKDIR /usr/share/nginx/html
COPY --from=build /app/tools/nginx/frontend-angular-logout.conf /etc/nginx/nginx.conf
COPY --from=build /app/dist/apps/frontend/angular/logout/browser .
RUN rm -f 50x.html
RUN rm -f 3rdpartylicenses.txt
ARG APP_PORT
ENV APP_PORT=$APP_PORT
EXPOSE $APP_PORT
CMD ["nginx", "-g", "daemon off;"]

FROM NEXUS_URL/nginx:stable-alpine-slim AS frontend-angular-register
WORKDIR /usr/share/nginx/html
COPY --from=build /app/tools/nginx/frontend-angular-register.conf /etc/nginx/nginx.conf
COPY --from=build /app/dist/apps/frontend/angular/register/browser .
RUN rm -f 50x.html
RUN rm -f 3rdpartylicenses.txt
ARG APP_PORT
ENV APP_PORT=$APP_PORT
EXPOSE $APP_PORT
CMD ["nginx", "-g", "daemon off;"]

FROM NEXUS_URL/nginx:stable-alpine-slim AS frontend-angular-shell
WORKDIR /usr/share/nginx/html
COPY --from=build /app/tools/nginx/frontend-angular-shell.conf /etc/nginx/nginx.conf
COPY --from=build /app/dist/apps/frontend/angular/shell/browser .
RUN rm -f 50x.html
RUN rm -f 3rdpartylicenses.txt
ARG APP_PORT
ENV APP_PORT=$APP_PORT
EXPOSE $APP_PORT
EXPOSE 4000
CMD ["nginx", "-g", "daemon off;"]

FROM NEXUS_URL/nginx:stable-alpine-slim AS frontend-angular-user
WORKDIR /usr/share/nginx/html
COPY --from=build /app/tools/nginx/frontend-angular-user.conf /etc/nginx/nginx.conf
COPY --from=build /app/dist/apps/frontend/angular/user/browser .
RUN rm -f 50x.html
RUN rm -f 3rdpartylicenses.txt
ARG APP_PORT
ENV APP_PORT=$APP_PORT
EXPOSE $APP_PORT
CMD ["nginx", "-g", "daemon off;"]

FROM NEXUS_URL/nginx:stable-alpine-slim AS frontend-angular-ui-storybook
WORKDIR /usr/share/nginx/html
COPY --from=build /app/tools/nginx/frontend-angular-ui-storybook.conf /etc/nginx/nginx.conf
COPY --from=build /app/dist/storybook/frontend/angular/ui/storybook .
RUN rm -f 50x.html
ARG APP_PORT
ENV APP_PORT=$APP_PORT
EXPOSE $APP_PORT
CMD ["nginx", "-g", "daemon off;"]

and seven of these app docker-compose.yml files

services:
    frontend-angular-dashboard:
        image: "NEXUS_URL/${FRONTEND_ANGULAR_DASHBOARD_APP_NAME}:${APP_TAG}"
        restart: always
        env_file:
            - .env
        build:
            context: ../../
            dockerfile: Dockerfile
            target: ${FRONTEND_ANGULAR_DASHBOARD_APP_NAME}
            args:
                - NPM_TOKEN=${NEXUS_NPM_AUTH}
                - NPM_REGISTRY=${NPM_PUBLIC_REGISTRY}
                - APP_PORT=${FRONTEND_ANGULAR_DASHBOARD_PORT}
        ports:
            - "${FRONTEND_ANGULAR_DASHBOARD_PORT}:${FRONTEND_ANGULAR_DASHBOARD_PORT}"
        networks:
            - frontend
kramar commented 1 month ago

The build works fine with version 19.2.2, which was our previous nx version. We're not sure from which version it stopped working.

danielritter commented 1 month ago

I got now the following error:

11:33:40   #56 [frontend-angular-logout build  5/14] RUN npm ci
11:33:40   #56 3463.2 npm error code 1
11:33:40   #56 3463.2 npm error path /app/node_modules/nx
11:33:40   #56 3463.2 npm error command failed
11:33:40   #56 3463.2 npm error command sh -c node ./bin/post-install
11:33:40   #56 3463.2 npm error /app/node_modules/nx/src/project-graph/plugins/isolation/plugin-pool.js:146
11:33:40   #56 3463.2 npm error                 const { resolver, rejector } = pending.get(tx);
11:33:40   #56 3463.2 npm error                         ^
11:33:40   #56 3463.2 npm error
11:33:40   #56 3463.2 npm error TypeError: Cannot destructure property 'resolver' of 'pending.get(...)' as it is undefined.
11:33:40   #56 3463.2 npm error     at createNodesResult (/app/node_modules/nx/src/project-graph/plugins/isolation/plugin-pool.js:146:25)
11:33:41   #56 3463.2 npm error     at consumeMessage (/app/node_modules/nx/src/project-graph/plugins/isolation/messaging.js:37:32)
11:33:41   #56 3463.2 npm error     at /app/node_modules/nx/src/project-graph/plugins/isolation/plugin-pool.js:73:47
11:33:41   #56 3463.2 npm error     at Socket.<anonymous> (/app/node_modules/nx/src/utils/consume-messages-from-socket.js:13:17)
11:33:41   #56 3463.2 npm error     at Socket.emit (node:events:519:28)
11:33:41   #56 3463.2 npm error     at addChunk (node:internal/streams/readable:559:12)
11:33:41   #56 3463.2 npm error     at readableAddChunkPushByteMode (node:internal/streams/readable:510:3)
11:33:41   #56 3463.2 npm error     at Readable.push (node:internal/streams/readable:390:5)
11:33:41   #56 3463.2 npm error     at Pipe.onStreamRead (node:internal/stream_base_commons:191:23)
11:33:41   #56 3463.2 npm error
11:33:41   #56 3463.2 npm error Node.js v20.15.1
11:33:41   #56 3463.3
11:33:41   #56 3463.3 npm error A complete log of this run can be found in: /home/node/.npm/_logs/2024-07-25T10_36_06_395Z-debug-0.log
11:33:43   #56 ERROR: process "/bin/sh -c npm ci" did not complete successfully: exit code: 1

I also updated to nx 19.5.3. I am not sure if it happens because of hardware resource issues but it should have enough RAM so I think it doesn't happen because of that.

Arilas commented 1 month ago

Has the same error. Made an a yarn patch with this changes and it's working:

diff --git a/src/project-graph/project-graph.js b/src/project-graph/project-graph.js
index cf1e507ff7cf801d9c09d7e415ef8c9d3d24aba6..95d61a31cc29a0170b7a7b13824d839db7f2700a 100644
--- a/src/project-graph/project-graph.js
+++ b/src/project-graph/project-graph.js
@@ -225,7 +225,7 @@ async function createProjectGraphAndSourceMapsAsync(opts = {
             return projectGraphAndSourceMaps;
         }
         catch (e) {
-            if (e.message.indexOf('inotify_add_watch') > -1) {
+            if (!e.message || e.message.indexOf('inotify_add_watch') > -1) {
                 // common errors with the daemon due to OS settings (cannot watch all the files available)
                 output_1.output.note({
                     title: `Unable to start Nx Daemon due to the limited amount of inotify watches, continuing without the daemon.`,

Seems that there's no message field in error and it's not executing job "without the daemon" as it suppose to do. But adding this condition trigger executing without the daemon

Jesse1989pp commented 1 month ago

I can bypass this error by turning off NX_DAEMON in my Dockerfiles; ENV NX_DAEMON=false.

I see @danielritter having it explicitly set to true in the Dockerfile. Normally this is disabled automatically within the CI, per https://nx.dev/concepts/nx-daemon#turning-it-off. Still, the above change mentioned might be the right thing to do.

danielritter commented 1 month ago

@Jesse1989pp Thanks for the hint, that really solved the issue for me. Setting ENV NX_DAEMON=false. Docker built successfully again. But I would also recommend to fix and do the the change mentioned above.

github-actions[bot] commented 1 month ago

This issue has been automatically marked as stale because no reproduction was provided within 7 days. Please help us help you. Providing a repository exhibiting the issue helps us diagnose and fix the issue. Any time that we spend reproducing this issue is time taken away from addressing this issue and other issues. This issue will be closed in 21 days if a reproduction is not provided. If a reproduction has been provided, please reply to keep it active. Thanks for being a part of the Nx community! 🙏