15.8.1 build failure on cross-platform development teams or CI/Docker pipelines

LongLiveCHIEF commented 1 year ago

Current Behavior

dependency installs fails due to inability to find module '@nrwl/nx-linux-x64-musl' in docker containers/pipelines.

Expected Behavior

Nx projects should install and build with same dependencies locally as in pipeline containers so dependencies can be frozen for change control/auditing purposes.

GitHub Repo

No response

Steps to Reproduce

Develop on mac or windows
install dependencies
run yarn install --ignore-optional --ignore-scripts --silent --frozen-lockfile in a docker container

Nx Report

>  NX   Report complete - copy this into the issue template

   Node : 18.12.1
   OS   : darwin arm64
   yarn : 1.22.19

   nx                      : 15.8.1
   @nrwl/js                : 15.8.1
   @nrwl/jest              : 15.8.1
   @nrwl/linter            : 15.8.1
   @nrwl/workspace         : 15.8.1
   @nrwl/cli               : 15.8.1
   @nrwl/cypress           : 15.8.1
   @nrwl/devkit            : 15.8.1
   @nrwl/eslint-plugin-nx  : 15.8.1
   @nrwl/node              : 15.8.1
   @nrwl/nx-plugin         : 15.8.1
   @nrwl/react             : 15.8.1
   @nrwl/rollup            : 15.8.1
   @nrwl/tao               : 15.8.1
   @nrwl/web               : 15.8.1
   @nrwl/webpack           : 15.8.1
   typescript              : 4.9.5
   ---------------------------------------
   Community plugins:
   @jscutlery/semver : 2.28.0
   ---------------------------------------
   Local workspace plugins:
         @my-project/nx

✨  Done in 1.24s.

Failure Logs

Step 9/16 : RUN yarn install --ignore-optional --ignore-scripts --silent --frozen-lockfile
 ---> Running in d5b6adb840d2
Removing intermediate container d5b6adb840d2
 ---> 5b36acfb8a4f
Step 10/16 : COPY . /app
 ---> c35c6da79680
Step 11/16 : RUN yarn nx build my-project
 ---> Running in c478b6c3918e
yarn run v1.22.19
$ /app/node_modules/.bin/nx build my-project
 >  NX   Cannot find module '@nrwl/nx-linux-x64-musl'
   Require stack:
   - /app/node_modules/nx/src/native/index.js
   - /app/node_modules/nx/src/hasher/native-file-hasher.js
   - /app/node_modules/nx/src/hasher/file-hasher.js
   - /app/node_modules/nx/src/project-graph/build-project-graph.js
   - /app/node_modules/nx/src/project-graph/project-graph.js
   - /app/node_modules/nx/src/utils/project-graph-utils.js
   - /app/node_modules/nx/src/tasks-runner/utils.js
   - /app/node_modules/nx/src/tasks-runner/run-command.js
   - /app/node_modules/nx/src/command-line/run-one.js
   - /app/node_modules/nx/src/command-line/nx-commands.js
   - /app/node_modules/nx/bin/init-local.js
   - /app/node_modules/nx/bin/nx.js
error Command failed with exit code 1.

Additional Information

This is probably related to the implementation of https://github.com/nrwl/nx/pull/15071, and may be a documentation issue, (no documentation was added for the

I am going to try setting NX_NON_NATIVE_HASHER=true in my docker/builds to see if this fixes, but i think there was a step missing in the design considerations for this feature.

Cammisuli commented 1 year ago

Can you try and add the native packages directly as dependencies in your package.json? Usually npm package managers will automatically install the optional dependencies based on criteria on those optional packages. And since you're running with --ignore-optional, it defeats that mechanism.

Add these (or whatever platforms you need to support/build on) to your dev/dependencies in package.json

"@nrwl/nx-darwin-arm64": "15.8.1",
"@nrwl/nx-darwin-x64": "15.8.1",
"@nrwl/nx-linux-arm-gnueabihf": "15.8.1",
"@nrwl/nx-linux-arm64-gnu": "15.8.1",
"@nrwl/nx-linux-arm64-musl": "15.8.1",
"@nrwl/nx-linux-x64-gnu": "15.8.1",
"@nrwl/nx-linux-x64-musl": "15.8.1",
"@nrwl/nx-win32-arm64-msvc": "15.8.1",
"@nrwl/nx-win32-x64-msvc": "15.8.1"

https://unpkg.com/nx@15.8.1/package.json

LongLiveCHIEF commented 1 year ago

I'm experimenting with removing the ignore-optional in our pipelines to see how this affects things. I think we had to add it to avoid failures in the first place though.

I had to make sure these weren't packages that had an associated binary, as that would also fail in our environments due to firewall rules (they install after testing).

Question... will these dependencies be updated along with the rest when doing a nx migrate latest command in the future if I add them as dependencies?

Cammisuli commented 1 year ago

actually, I dont think adding them as deps will help because package managers check the cpu architecture.. let me think of something

Cammisuli commented 1 year ago

hmm, it looks like removing --ignore-optional would be the best bet at the moment. The lock file contains all the integrity shas for all the optionals, so those can still be audited.

LongLiveCHIEF commented 1 year ago

yeah, removing --ignore-optional does work, at least with the single project i tested the build for. What would be the best documentation approach for this and the NX_NON_NATIVE_HASHER=true.

ertl commented 1 year ago

Experiencing a similar problem since the new update, but I don't have a --ignore-optional flag.. Getting the following error:

\?\C:\builds\kixomatic\kixomatic\node_modules\@nrwl\nx-win32-x64-msvc\nx.win32-x64-msvc.node ERROR: Job failed: exit code 1

Any suggestions to solve this?

Cammisuli commented 1 year ago

@LongLiveCHIEF I'm going to update the docs to mention how to opt out of the native code implementations.

@ertl can you provide more information on your environment? What version of Windows, how are you building it?

LongLiveCHIEF commented 1 year ago

@ertl also might be helpful to report what terminal you run your commands in (git bash/mingw i'm guessing?). I have a hunch for this one due to how node operates on windows.

ertl commented 1 year ago

@Cammisuli The build runs in a windows docker image (mcr.microsoft.com/windows/servercore:20H2) in a gitlab CI/CD Pipeline.

The Image uses the latest Node 16 Version with the cmd.exe as terminal

As package manager we use npm

ertl commented 1 year ago

@Cammisuli @LongLiveCHIEF: looks like setting NX_NON_NATIVE_HASHER=true fixes my problem

pavelbinar commented 1 year ago

Hi, I have a similar problem Error: Cannot find module '@nrwl/nx-linux-x64-musl' running docker build on the CI (DigitalOcean App platform).

Dockerfile

FROM node:18.14.1-alpine as build-system
WORKDIR /app
COPY package*.json ./
RUN npm install
COPY . .
RUN nx run-many --target=build

"@nrwl/cli": "15.8.5"
node:18.14.1-alpine
not using --ignore-optional
NX_NON_NATIVE_HASHER=true did not help

arrocke commented 1 year ago

I was able to fix this by adding @nrwl/nx-linux-x64-gnu to my optional dependencies. That allowed it to be installed on CI, but not break the install in my local dev environment.

pavelbinar commented 1 year ago

Thank you @arrocke ! Yes, adding this to package.json helped:

"optionalDependencies": {
    "@nrwl/nx-linux-x64-musl": "^15.8.5"
  }

I am adding -x64-musl since this is apperatnly used on the alpine linux.

@Cammisuli is this a temporary fix or something that shall stay?

Cammisuli commented 1 year ago

We shouldn't have to manually add optional dependencies to the main package.json. Those are already set in the nx package

What's the version of npm being used?

pavelbinar commented 1 year ago

All right, thank you for your answer.

node v18.14.1
npm 9.3.1

BTW: you shall be able to replicate it by using docker with the following image:

FROM node:18.14.1-alpine

Cammisuli commented 1 year ago

@pavelbinar it appears to be working for me..

Here's what I did to build the image

Then connecting to the instance, I can see the right packages in the @nrwl namespace in node_modules

pavelbinar commented 1 year ago

Thank you, @Cammisuli for trying to resolve this problem! I still do not see the package inside the image. Here is what I am using to test it:

here is the content of Dockerfile:

FROM node:18.14.1-alpine
WORKDIR /app
COPY package*.json ./
RUN npm install
RUN ls -d node_modules/nx*/

docker build . -t testing --progress=plain --no-cache --platform linux/amd64
#1 [internal] load build definition from Dockerfile
#1 sha256:c60ebfa77bb0c255c4bf039800821e79f9ed127e7b5917d9168436072241c91b
#1 transferring dockerfile: 228B done
#1 DONE 0.0s

#2 [internal] load .dockerignore
#2 sha256:1248faf17fb6e753daf5dbcdea50cee4242a4403fb0892f8c1d8aeb336857315
#2 transferring context: 35B done
#2 DONE 0.0s

#3 [internal] load metadata for docker.io/library/node:18.14.1-alpine
#3 sha256:4f4b31ec22f05d3e4aff615d4fc09a97cebee998fc6cd136a853767b34a35b09
#3 DONE 0.7s

#4 [1/5] FROM docker.io/library/node:18.14.1-alpine@sha256:e0a779479fca9cf43cae2852291bee70e730ae3ad27fea1211060bd344b696b8
#4 sha256:34ea6150a404a947870f9ef41d5ad895932859fccf96472fb391e0f32535504f
#4 DONE 0.0s

#5 [2/5] WORKDIR /app
#5 sha256:253858d1948a7c30557fbf9e1f5ee06be0aaee31c3dc301c810156a8f6f6021b
#5 CACHED

#6 [internal] load build context
#6 sha256:07d9588fa6c5c6227e76d3f7d676a123f1671c7a8ca5fdc533891024d7e8d963
#6 transferring context: 34B done
#6 DONE 0.0s

#7 [3/5] COPY package*.json ./
#7 sha256:de89b53a98249277d5c52bd73871392735486669aed4c74b98a316c8ea279a51
#7 DONE 0.0s

#8 [4/5] RUN npm install
#8 sha256:a4f48f2984d27239ea162955d80e6fa71bf5ca1b6a504b63b876e0a5ba10f509
#8 119.1 npm WARN ERESOLVE overriding peer dependency
#8 343.3 npm WARN deprecated stable@0.1.8: Modern JS already guarantees Array#sort() is a stable sort, so this library is deprecated. See the compatibility table on MDN: https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Array/sort#browser_compatibility
#8 343.5 npm WARN deprecated source-map-resolve@0.6.0: See https://github.com/lydell/source-map-resolve#deprecated
#8 348.3 npm WARN deprecated sourcemap-codec@1.4.8: Please use @jridgewell/sourcemap-codec instead
#8 372.6 npm WARN deprecated svgo@1.3.2: This SVGO version is no longer supported. Upgrade to v2.x.x.
#8 397.8 npm WARN deprecated popper.js@1.16.1: You can find the new Popper v2 at @popperjs/core, this package is dedicated to the legacy v1
#8 463.5
#8 463.5 added 2360 packages, and audited 2361 packages in 8m
#8 463.5
#8 463.5 284 packages are looking for funding
#8 463.5   run `npm fund` for details
#8 463.5
#8 463.5 7 vulnerabilities (2 moderate, 5 high)
#8 463.5
#8 463.5 To address all issues (including breaking changes), run:
#8 463.5   npm audit fix --force
#8 463.5
#8 463.5 Run `npm audit` for details.
#8 463.5 npm notice
#8 463.5 npm notice New minor version of npm available! 9.3.1 -> 9.6.1
#8 463.5 npm notice Changelog: <https://github.com/npm/cli/releases/tag/v9.6.1>
#8 463.5 npm notice Run `npm install -g npm@9.6.1` to update!
#8 463.5 npm notice
#8 DONE 466.8s

#9 [5/5] RUN ls -d node_modules/nx*/
#9 sha256:4d86d55109ef19e0042505c19e59521df99f3fb681607430d972056c65eea1be
#9 0.335 node_modules/nx/
#9 DONE 0.4s

#10 exporting to image
#10 sha256:e8c613e07b0b7ff33893b694f7759a10d42e180f2b4dc349fb57dc6b71dcab00
#10 exporting layers
#10 exporting layers 29.4s done
#10 writing image sha256:2af49d419f4bcffea5fa8fc8b54b262ce14f20f719c196b30b1872bf33b1ad43
#10 writing image sha256:2af49d419f4bcffea5fa8fc8b54b262ce14f20f719c196b30b1872bf33b1ad43 done
#10 naming to docker.io/library/testing done
#10 DONE 29.4s

BTW: I tried to delete package-lock.json, but no difference.

Cammisuli commented 1 year ago

@pavelbinar the native packages should be under node_modules/@nrwl

Here's my output using the same command as you (with the ls pointint to node_modules/@nrwl/nx*/:

docker build . -t 15380 --progress=plain --no-cache --platform linux/amd64
#1 [internal] load build definition from Dockerfile
#1 sha256:2a633d1a6ad1719a81fee1a722992df015ddee1597988ffd6737814c3cd0732a
#1 transferring dockerfile: 36B done
#1 DONE 0.0s

#2 [internal] load .dockerignore
#2 sha256:74cb9da7184885db77548373c6c5ac727f0ca9c7e5468d6b32abae865697fac6
#2 transferring context: 2B done
#2 DONE 0.0s

#3 [internal] load metadata for docker.io/library/node:18.14.1-alpine
#3 sha256:4f4b31ec22f05d3e4aff615d4fc09a97cebee998fc6cd136a853767b34a35b09
#3 DONE 0.8s

#4 [1/5] FROM docker.io/library/node:18.14.1-alpine@sha256:e0a779479fca9cf43cae2852291bee70e730ae3ad27fea1211060bd344b696b8
#4 sha256:34ea6150a404a947870f9ef41d5ad895932859fccf96472fb391e0f32535504f
#4 DONE 0.0s

#5 [2/5] WORKDIR /app
#5 sha256:253858d1948a7c30557fbf9e1f5ee06be0aaee31c3dc301c810156a8f6f6021b
#5 CACHED

#6 [internal] load build context
#6 sha256:435cd8d0b1b6361e769e08d3315de1ee4d4c5c9297bc2bb6d10213792de5cb34
#6 transferring context: 146B done
#6 DONE 0.0s

#7 [3/5] COPY package*.json ./
#7 sha256:898ff4f239e8f6d1fa4fb7b99efc27aeb00055d20adb45807d47516f6ca94d03
#7 DONE 0.0s

#8 [4/5] RUN npm install
#8 sha256:12e405c62361e26c0231237cd3731163af70c1c24273ae686091e580c1a36275
#8 25.63 npm WARN deprecated stable@0.1.8: Modern JS already guarantees Array#sort() is a stable sort, so this library is deprecated. See the compatibility table on MDN: https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Array/sort#browser_compatibility
#8 26.53 npm WARN deprecated source-map-resolve@0.6.0: See https://github.com/lydell/source-map-resolve#deprecated
#8 72.13 
#8 72.13 > monodon@0.0.0 prepare
#8 72.13 > husky install
#8 72.13 
#8 72.64 husky - git command not found, skipping install
#8 72.77 
#8 72.77 added 1338 packages, and audited 1339 packages in 1m
#8 72.77 
#8 72.77 137 packages are looking for funding
#8 72.77   run `npm fund` for details
#8 72.78 
#8 72.78 found 0 vulnerabilities
#8 72.78 npm notice 
#8 72.78 npm notice New minor version of npm available! 9.3.1 -> 9.6.1
#8 72.78 npm notice Changelog: <https://github.com/npm/cli/releases/tag/v9.6.1>
#8 72.78 npm notice Run `npm install -g npm@9.6.1` to update!
#8 72.78 npm notice 
#8 DONE 73.4s

#9 [5/5] RUN ls -d node_modules/@nrwl/nx*/
#9 sha256:15c0f4f487734b78aa88afaa0fb24f7be5bdc864fa70da42db18506feadd9415
#9 0.332 node_modules/@nrwl/nx-cloud/
#9 0.332 node_modules/@nrwl/nx-linux-x64-gnu/
#9 0.332 node_modules/@nrwl/nx-linux-x64-musl/
#9 0.332 node_modules/@nrwl/nx-plugin/
#9 DONE 0.3s

#10 exporting to image
#10 sha256:e8c613e07b0b7ff33893b694f7759a10d42e180f2b4dc349fb57dc6b71dcab00
#10 exporting layers
#10 exporting layers 2.1s done
#10 writing image sha256:5c51c0e05af0e19b667d08360937d2c8529a4c536c042c651cadc32a0794209d done
#10 naming to docker.io/library/15380 done
#10 DONE 2.1s

Use 'docker scan' to run Snyk tests against images to find vulnerabilities and learn how to fix them

The sha's are mostly the same other then the actual workspace details.

Is there a way for me to pull the workspace to take a look?

tamaserdfalvi commented 1 year ago

As requested here this is our package.json :

{
    "name": "tcp",
    "version": "23.1.41",
    "license": "MIT",
    "scripts": {
        "audit": "npm audit --omit=dev",
        "bom": "cyclonedx-npm --omit dev --short-PURLs --output-file bom.json --mc-type=application --package-lock-only --ignore-npm-errors",
        "postinstall": "ngcc --properties es2015 browser module main"
    },
    "private": true,
    "cypress-cucumber-preprocessor": {
        "nonGlobalStepDefinitions": false,
        "stepDefinitions": "../../../apps/frontend/test/step_definitions"
    },
    "devDependencies": {
        "@angular-devkit/build-angular": "15.2.1",
        "@angular-eslint/eslint-plugin": "15.2.1",
        "@angular-eslint/eslint-plugin-template": "15.2.1",
        "@angular-eslint/template-parser": "15.2.1",
        "@angular/cli": "~15.2.0",
        "@angular/compiler-cli": "15.2.1",
        "@angular/language-service": "15.2.1",
        "@cyclonedx/bom": "^4.0.3",
        "@cypress/browserify-preprocessor": "^3.0.2",
        "@nrwl/angular": "15.8.3",
        "@nrwl/cypress": "15.8.3",
        "@nrwl/esbuild": "15.8.3",
        "@nrwl/eslint-plugin-nx": "15.8.3",
        "@nrwl/jest": "15.8.3",
        "@nrwl/js": "15.8.3",
        "@nrwl/linter": "15.8.3",
        "@nrwl/node": "15.8.3",
        "@nrwl/nx-cloud": "15.1.1",
        "@nrwl/webpack": "15.8.3",
        "@nrwl/workspace": "15.8.3",
        "@types/amqplib": "^0.10.1",
        "@types/camunda-external-task-client-js": "^1.3.3",
        "@types/express": "4.17.13",
        "@types/faker": "^4.1.7",
        "@types/file-saver": "^2.0.5",
        "@types/form-data": "^2.5.0",
        "@types/fs-extra": "^11.0.1",
        "@types/glob": "^7.1.3",
        "@types/hapi__joi": "^17.1.9",
        "@types/jest": "29.4.0",
        "@types/mime-types": "^2.1.1",
        "@types/mock-req-res": "^1.1.3",
        "@types/moment": "^2.13.0",
        "@types/morgan": "^1.9.4",
        "@types/multer": "^1.4.7",
        "@types/node": "18.7.1",
        "@types/nodemailer": "^6.4.7",
        "@types/openpgp": "^4.4.18",
        "@types/papaparse": "^5.3.2",
        "@types/pdfkit": "^0.12.8",
        "@types/sinon": "^7.5.2",
        "@types/swagger-jsdoc": "^6.0.1",
        "@types/tedious": "^4.0.9",
        "@types/tinycolor2": "1.4.2",
        "@types/uuid": "^8.3.0",
        "@typescript-eslint/eslint-plugin": "~5.29.0",
        "@typescript-eslint/parser": "~5.29.0",
        "coverage-badges": "^1.0.7",
        "cypress": "^12.5.1",
        "cypress-cucumber-preprocessor": "^4.3.1",
        "cypress-multi-reporters": "^1.6.1",
        "cypress-real-events": "^1.7.2",
        "esbuild": "0.17.5",
        "eslint": "~8.12.0",
        "eslint-config-prettier": "8.1.0",
        "eslint-plugin-cypress": "^2.10.3",
        "express": "^4.18.1",
        "faker": "^4.1.0",
        "jest": "29.4.3",
        "jest-coverage-badges": "^1.1.2",
        "jest-environment-jsdom": "29.4.3",
        "jest-environment-node": "^29.4.1",
        "jest-mock-req-res": "^1.0.2",
        "keycloak-mock": "^1.0.10",
        "mochawesome": "^7.1.3",
        "mochawesome-merge": "^4.2.2",
        "mochawesome-report-generator": "^6.2.0",
        "mock-req-res": "^1.2.1",
        "nx": "15.8.3",
        "prettier": "^2.6.2",
        "start-server-and-test": "^1.14.0",
        "ts-jest": "29.0.5",
        "ts-node": "10.9.1",
        "tslint": "~6.1.0",
        "typescript": "^4.9.5"
    },
    "dependencies": {
        "@angular-material-extensions/fab-menu": "^6.1.0",
        "@angular/animations": "15.2.1",
        "@angular/cdk": "15.2.1",
        "@angular/common": "15.2.1",
        "@angular/compiler": "15.2.1",
        "@angular/core": "15.2.1",
        "@angular/elements": "15.2.1",
        "@angular/flex-layout": "^15.0.0-beta.42",
        "@angular/forms": "15.2.1",
        "@angular/localize": "15.2.1",
        "@angular/material": "15.2.1",
        "@angular/platform-browser": "15.2.1",
        "@angular/platform-browser-dynamic": "15.2.1",
        "@angular/router": "15.2.1",
        "@hapi/joi": "^17.1.1",
        "@json2csv/plainjs": "^6.1.2",
        "@kubernetes/client-node": "^0.17.1",
        "@ngneat/transloco": "^3.2.0",
        "amqplib": "^0.10.3",
        "angular-gridster2": "^15.0.3",
        "axios": "^1.1.3",
        "body-parser": "^1.20.1",
        "bootstrap": "^4.6.2",
        "camunda-external-task-client-js": "^2.3.1",
        "camunda-worker-node": "^7.1.0",
        "compression": "^1.7.4",
        "cors": "^2.8.5",
        "dmn-evaluator": "^0.1.0",
        "directory-tree": "^2.2.5",
        "dotenv": "^16.0.3",
        "email-validator": "^2.0.4",
        "express-session": "^1.17.3",
        "express-ws": "^5.0.2",
        "file-saver": "^2.0.5",
        "filesize": "^10.0.6",
        "form-data": "^4.0.0",
        "fs-extra": "^10.1.0",
        "glob": "^7.1.6",
        "handlebars": "^4.7.7",
        "hot-formula-parser": "^4.0.0",
        "html2canvas": "^1.4.1",
        "jquery": "^3.6.3",
        "json2csv": "^4.5.1",
        "jsonpath-plus": "^7.2.0",
        "jspdf": "^2.5.1",
        "keycloak-angular": "^13.0.0",
        "keycloak-connect": "^21.0.1",
        "keycloak-js": "^18.0.1",
        "lodash.merge": "^4.6.2",
        "mime-types": "^2.1.34",
        "moment": "^2.29.1",
        "moment-timezone": "^0.5.40",
        "mongoose": "^6.7.5",
        "morgan": "^1.10.0",
        "multer": "^1.4.5-lts.1",
        "mysql2": "^2.3.3",
        "ngx-cookie": "6.0.0",
        "ngx-mat-select-search": "^7.0.0",
        "ngx-socket-io": "^4.4.0",
        "nodemailer": "^6.9.1",
        "oauth": "^0.9.15",
        "openpgp": "^4.10.10",
        "papaparse": "^5.3.2",
        "pdfkit": "^0.13.0",
        "pdfkit-table": "^0.1.99",
        "perfect-express-sanitizer": "^1.0.11",
        "qs": "^6.11.0",
        "request": "^2.88.2",
        "rxjs": "^7.8.0",
        "socket.io": "^4.4.1",
        "socket.io-client": "^4.4.1",
        "swagger-jsdoc": "^6.2.8",
        "swagger-ui-dist": "^4.17.1",
        "tedious": "^15.1.3",
        "tinycolor2": "1.4.2",
        "translation-json-csv-converter": "github:mfodor/translation-json-csv-converter",
        "ts-md5": "^1.2.4",
        "tslib": "^2.3.0",
        "uuid": "^8.3.2",
        "winston": "^3.8.2",
        "xlsx": "^0.18.5",
        "zone.js": "0.12.0"
    }
}

Cammisuli commented 1 year ago

@tamaserdfalvi nothing stands out to me with your package.json.

What does your bitbucket flow look like? I really want to know how node_modules gets installed, and with what command.

And if possible could you do a ls -d node_modules/@nrwl/nx*/ after the node_modules install and tell me what you see?

tamaserdfalvi commented 1 year ago

The pipeline is pretty complex, but this can be replicated anytime even with this simple step :

      - step:
          caches:
            - node
            - cypress-cache
          name: Install project
          runs-on:
            - self.hosted
            - linux
          script:
            - export NODE_ENV=development
            - npm ci
            - ls -d node_modules/@nrwl/nx*/
            - npx nx affected -t test --base=master
            - npx cypress install

It does NOT make difference if we're not using our own runners, but bitbucket's own, I've tried it. Here's the log of that step failing, that contains the directory list you've asked for too : pipelineLog-{02ddb5d7-02ca-45e8-a318-41ef93096891}.txt

Cammisuli commented 1 year ago

@tamaserdfalvi whats the version of node and npm for that instance?

tamaserdfalvi commented 1 year ago

We're using the image cypress/base:18.14.1 which has Node 18.14.1

deminoth commented 1 year ago

~Any updates? My team is using Intel and Apple Silicon macs and we are having the same problem.~ The issue has gone but I don't know why 😞

harmoney-franck commented 1 year ago

Hi, same problem here with pnpm v8, on a node:18-alpine3.16, any update?

Brooooooklyn commented 1 year ago

Hi, same problem here with pnpm v8, on a node:18-alpine3.16, any update?

@harmoney-franck could you please try node:18-alpine ?

Thank you @arrocke ! Yes, adding this to package.json helped:
"optionalDependencies": {
    "@nrwl/nx-linux-x64-musl": "^15.8.5"
  }
I am adding -x64-musl since this is apperatnly used on the alpine linux.

@Cammisuli is this a temporary fix or something that shall stay?

Your issue is most likely related to https://github.com/npm/cli/issues/4828

harmoney-franck commented 1 year ago

Hi, same problem here with pnpm v8, on a node:18-alpine3.16, any update?

@harmoney-franck could you please try node:18-alpine ?
Thank you @arrocke ! Yes, adding this to package.json helped:
"optionalDependencies": {
    "@nrwl/nx-linux-x64-musl": "^15.8.5"
  }
I am adding -x64-musl since this is apperatnly used on the alpine linux. @Cammisuli is this a temporary fix or something that shall stay?
Your issue is most likely related to npm/cli#4828

It seems to work with this image, but still getting the error in the console

Brooooooklyn commented 1 year ago

@harmoney-franck what kind of error, do you have any messages or screenshot?

harmoney-franck commented 1 year ago

@Brooooooklyn warnings not errors sorry

Brooooooklyn commented 1 year ago

@Brooooooklyn warnings not errors sorry

@harmoney-franck your warning was from next.js, not the nx.

github-actions[bot] commented 1 year ago

This issue has been closed for more than 30 days. If this issue is still occuring, please open a new issue with more recent context.

nrwl / nx