npm preinstall no longer working

[dbburgess]

👋 Hello. It seems a recent change has caused our preinstall script to not run in time.

Our dependencies include private packages which we need to run an auth step for, which we were using the npm preinstall step to ensure credentials were configured before the install which tries to install the private packages. It seems a recent change has broken this behavior. Any suggestions, or other ways to achieve this?

Successful Build Output

Using `gcr.io/google-appengine/nodejs@sha256:1b9d17453000c2c6c819b055688821db3deaa2dfa7fb4e73606c8788f6c9e29a` (tagged `2022-04-02-002411`): ``` starting build "{redacted}" FETCHSOURCE BUILD Starting Step #0 - "fetcher" Step #0 - "fetcher": Already have image (with digest): gcr.io/cloud-builders/gcs-fetcher Step #0 - "fetcher": Fetching manifest gs://{redacted}.appspot.com/ae/{redacted}/manifest.json. Step #0 - "fetcher": Processing 84 files. Step #0 - "fetcher": ****************************************************** Step #0 - "fetcher": Status: SUCCESS Step #0 - "fetcher": Started: 2022-11-03T14:43:44Z Step #0 - "fetcher": Completed: 2022-11-03T14:43:44Z Step #0 - "fetcher": Requested workers: 200 Step #0 - "fetcher": Actual workers: 84 Step #0 - "fetcher": Total files: 84 Step #0 - "fetcher": Total retries: 0 Step #0 - "fetcher": GCS timeouts: 0 Step #0 - "fetcher": MiB downloaded: 1.20 MiB Step #0 - "fetcher": MiB/s throughput: 8.08 MiB/s Step #0 - "fetcher": Time for manifest: 63.51 ms Step #0 - "fetcher": Total time: 0.22 s Step #0 - "fetcher": ****************************************************** Finished Step #0 - "fetcher" Starting Step #1 Step #1: Pulling image: gcr.io/gcp-runtimes/nodejs/gen-dockerfile@sha256:8e2b8eef87fae1bc4b80206cc84139a52973433daafef5ba30ae84f2f85d01b6 Step #1: gcr.io/gcp-runtimes/nodejs/gen-dockerfile@sha256:8e2b8eef87fae1bc4b80206cc84139a52973433daafef5ba30ae84f2f85d01b6: Pulling from gcp-runtimes/nodejs/gen-dockerfile Step #1: Digest: sha256:8e2b8eef87fae1bc4b80206cc84139a52973433daafef5ba30ae84f2f85d01b6 Step #1: Status: Downloaded newer image for gcr.io/gcp-runtimes/nodejs/gen-dockerfile@sha256:8e2b8eef87fae1bc4b80206cc84139a52973433daafef5ba30ae84f2f85d01b6 Step #1: gcr.io/gcp-runtimes/nodejs/gen-dockerfile@sha256:8e2b8eef87fae1bc4b80206cc84139a52973433daafef5ba30ae84f2f85d01b6 Step #1: Checking for Node.js. Finished Step #1 Starting Step #2 Step #2: Already have image (with digest): gcr.io/kaniko-project/executor@sha256:f87c11770a4d3ed33436508d206c584812cd656e6ed08eda1cff5c1ee44f5870 Step #2: INFO[0000] Removing ignored files from build context: [node_modules .dockerignore Dockerfile npm-debug.log yarn-error.log .git .hg .svn app.yaml] Step #2: INFO[0000] Downloading base image gcr.io/google-appengine/nodejs@sha256:1b9d17453000c2c6c819b055688821db3deaa2dfa7fb4e73606c8788f6c9e29a Step #2: INFO[0016] Taking snapshot of full filesystem... Step #2: INFO[0024] Using files from context: [/workspace] Step #2: INFO[0024] COPY . /app/ Step #2: INFO[0024] RUN NODE_ENV=development npm install --unsafe-perm || ((if [ -f npm-debug.log ]; then cat npm-debug.log; fi) && false) Step #2: INFO[0024] cmd: /bin/sh Step #2: INFO[0024] args: [-c NODE_ENV=development npm install --unsafe-perm || ((if [ -f npm-debug.log ]; then cat npm-debug.log; fi) && false)] Step #2: Step #2: > app@0.1.0 preinstall /app ... ``` Past this point, everything went fine with the build.

Failing Build Output

Using `gcr.io/google-appengine/nodejs@sha256:48c8ffe761db6b81aaae36997b4934537ea84ff82c59aafc6c7fa2589b23ed1d` (tagged `latest` `2022-11-04-204206`): ``` starting build "{redacted}" FETCHSOURCE BUILD Starting Step #0 - "fetcher" Step #0 - "fetcher": Already have image (with digest): gcr.io/cloud-builders/gcs-fetcher Step #0 - "fetcher": Fetching manifest gs://{redacted}.appspot.com/ae/{redacted}/manifest.json. Step #0 - "fetcher": Processing 84 files. Step #0 - "fetcher": ****************************************************** Step #0 - "fetcher": Status: SUCCESS Step #0 - "fetcher": Started: 2022-11-14T19:28:33Z Step #0 - "fetcher": Completed: 2022-11-14T19:28:33Z Step #0 - "fetcher": Requested workers: 200 Step #0 - "fetcher": Actual workers: 84 Step #0 - "fetcher": Total files: 84 Step #0 - "fetcher": Total retries: 0 Step #0 - "fetcher": GCS timeouts: 0 Step #0 - "fetcher": MiB downloaded: 1.20 MiB Step #0 - "fetcher": MiB/s throughput: 9.71 MiB/s Step #0 - "fetcher": Time for manifest: 54.46 ms Step #0 - "fetcher": Total time: 0.18 s Step #0 - "fetcher": ****************************************************** Finished Step #0 - "fetcher" Starting Step #1 Step #1: Pulling image: gcr.io/gcp-runtimes/nodejs/gen-dockerfile@sha256:82b343146ebfe0336f87884e404f1602ab57d87929c1dc554bf1cecb544111c0 Step #1: gcr.io/gcp-runtimes/nodejs/gen-dockerfile@sha256:82b343146ebfe0336f87884e404f1602ab57d87929c1dc554bf1cecb544111c0: Pulling from gcp-runtimes/nodejs/gen-dockerfile Step #1: 6c5b97b864a6: Already exists Step #1: 3bc468948ab5: Already exists Step #1: 3c2cba919283: Already exists Step #1: 9d782453afdd: Pulling fs layer Step #1: 298f6c1305a7: Pulling fs layer Step #1: 0b07672deb61: Pulling fs layer Step #1: 8c9487bb2306: Pulling fs layer Step #1: 17204c73e510: Pulling fs layer Step #1: f80fe3d30525: Pulling fs layer Step #1: 9d278df2ce01: Pulling fs layer Step #1: 9fffe8f3b04d: Pulling fs layer Step #1: c9fe7cd32a7f: Pulling fs layer Step #1: 8c9487bb2306: Waiting Step #1: 17204c73e510: Waiting Step #1: f80fe3d30525: Waiting Step #1: 9d278df2ce01: Waiting Step #1: 9fffe8f3b04d: Waiting Step #1: c9fe7cd32a7f: Waiting Step #1: 0b07672deb61: Download complete Step #1: 298f6c1305a7: Verifying Checksum Step #1: 298f6c1305a7: Download complete Step #1: 8c9487bb2306: Verifying Checksum Step #1: 8c9487bb2306: Download complete Step #1: f80fe3d30525: Verifying Checksum Step #1: f80fe3d30525: Download complete Step #1: 9d278df2ce01: Verifying Checksum Step #1: 9d278df2ce01: Download complete Step #1: 9fffe8f3b04d: Verifying Checksum Step #1: 9fffe8f3b04d: Download complete Step #1: 17204c73e510: Verifying Checksum Step #1: 17204c73e510: Download complete Step #1: 9d782453afdd: Verifying Checksum Step #1: 9d782453afdd: Download complete Step #1: c9fe7cd32a7f: Verifying Checksum Step #1: c9fe7cd32a7f: Download complete Step #1: 9d782453afdd: Pull complete Step #1: 298f6c1305a7: Pull complete Step #1: 0b07672deb61: Pull complete Step #1: 8c9487bb2306: Pull complete Step #1: 17204c73e510: Pull complete Step #1: f80fe3d30525: Pull complete Step #1: 9d278df2ce01: Pull complete Step #1: 9fffe8f3b04d: Pull complete Step #1: c9fe7cd32a7f: Pull complete Step #1: Digest: sha256:82b343146ebfe0336f87884e404f1602ab57d87929c1dc554bf1cecb544111c0 Step #1: Status: Downloaded newer image for gcr.io/gcp-runtimes/nodejs/gen-dockerfile@sha256:82b343146ebfe0336f87884e404f1602ab57d87929c1dc554bf1cecb544111c0 Step #1: gcr.io/gcp-runtimes/nodejs/gen-dockerfile@sha256:82b343146ebfe0336f87884e404f1602ab57d87929c1dc554bf1cecb544111c0 Step #1: Checking for Node.js. Finished Step #1 Starting Step #2 Step #2: Already have image (with digest): gcr.io/kaniko-project/executor@sha256:f87c11770a4d3ed33436508d206c584812cd656e6ed08eda1cff5c1ee44f5870 Step #2: INFO[0000] Removing ignored files from build context: [node_modules .dockerignore Dockerfile npm-debug.log yarn-error.log .git .hg .svn app.yaml] Step #2: INFO[0000] Downloading base image gcr.io/google-appengine/nodejs@sha256:48c8ffe761db6b81aaae36997b4934537ea84ff82c59aafc6c7fa2589b23ed1d Step #2: INFO[0017] Taking snapshot of full filesystem... Step #2: INFO[0024] Using files from context: [/workspace] Step #2: INFO[0024] COPY . /app/ Step #2: INFO[0024] RUN /usr/local/bin/install_node '16.14.x' Step #2: INFO[0024] cmd: /bin/sh Step #2: INFO[0024] args: [-c /usr/local/bin/install_node '16.14.x'] Step #2: % Total % Received % Xferd Average Speed Time Time Time Current Step #2: Dload Upload Total Spent Left Speed Step #2: 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 100 31.3M 100 31.3M 0 0 66.9M 0 --:--:-- --:--:-- --:--:-- 66.8M Step #2: % Total % Received % Xferd Average Speed Time Time Time Current Step #2: Dload Upload Total Spent Left Speed Step #2: 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 100 4035 100 4035 0 0 27836 0 --:--:-- --:--:-- --:--:-- 28020 Step #2: node-v16.14.2-linux-x64.tar.gz: OK Step #2: Installed Node.js v16.14.2 Step #2: INFO[0027] RUN NODE_ENV=development npm install --unsafe-perm || ((if [ -f npm-debug.log ]; then cat npm-debug.log; fi) && false) Step #2: INFO[0027] cmd: /bin/sh Step #2: INFO[0027] args: [-c NODE_ENV=development npm install --unsafe-perm || ((if [ -f npm-debug.log ]; then cat npm-debug.log; fi) && false)] Step #2: npm WARN EBADENGINE Unsupported engine { Step #2: npm WARN EBADENGINE package: 'app@0.1.0', Step #2: npm WARN EBADENGINE required: { node: '16.14.x', npm: '8.11.x' }, Step #2: npm WARN EBADENGINE current: { node: 'v16.14.2', npm: '8.5.0' } Step #2: npm WARN EBADENGINE } Step #2: npm WARN deprecated uuid@3.4.0: Please upgrade to version 7 or higher. Older versions may use Math.random() in certain circumstances, which is known to be problematic. See https://v8.dev/blog/math-random for details. Step #2: npm notice Step #2: npm notice New major version of npm available! 8.5.0 -> 9.1.1 Step #2: npm notice Changelog: Step #2: npm notice Run `npm install -g npm@9.1.1` to update! Step #2: npm notice Step #2: npm ERR! code E403 Step #2: npm ERR! 403 403 Forbidden - GET https://us-east4-npm.pkg.dev/{redacted}/{redacted}/{redacted}/{redacted}/-/{redacted}/package-0.0.16.tgz - Permission "artifactregistry.repositories.downloadArtifacts" denied on resource "projects/{redacted}/locations/us-east4/repositories/{redacted}" (or it may not exist) Step #2: npm ERR! 403 In most cases, you or one of your dependencies are requesting Step #2: npm ERR! 403 a package version that is forbidden by your security policy, or Step #2: npm ERR! 403 on a server you do not have access to. Step #2: Step #2: npm ERR! A complete log of this run can be found in: Step #2: npm ERR! /root/.npm/_logs/2022-11-14T19_29_21_649Z-debug-0.log Step #2: error building image: error building stage: waiting for process to exit: exit status 1 Finished Step #2 ERROR ERROR: build step 2 "gcr.io/kaniko-project/executor@sha256:f87c11770a4d3ed33436508d206c584812cd656e6ed08eda1cff5c1ee44f5870" failed: step exited with non-zero status: 1 ```

Nothing has changed in the source code, dependencies, permissions, etc. Running the exact same code from the successful build yields a failing build. I suspect this is related to this change https://github.com/GoogleCloudPlatform/nodejs-docker/pull/260, which altered the order in which certain steps happen, but I'm not familiar enough to be certain. Any insight?

[kennethye1]

Does changing your npm to 8.11.x fix this? According to the logs

Step #2: npm WARN EBADENGINE Unsupported engine {
Step #2: npm WARN EBADENGINE   package: 'app@0.1.0',
Step #2: npm WARN EBADENGINE   required: { node: '16.14.x', npm: '8.11.x' },
Step #2: npm WARN EBADENGINE   current: { node: 'v16.14.2', npm: '8.5.0' }
Step #2: npm WARN EBADENGINE }

I'll have to investigate further.

[kennethye1]

Never mind, I've rolled back and will try to investigate.

[dbburgess]

Thank you, I really appreciate that! Please let me know if I can do anything to help.

[kennethye1]

What is your scripts and engines configuration in your package.json? Specifically the gcp-build.

[dbburgess]

Engines is:

{
  "node": "16.14.x",
  "npm": "8.11.x"
}

Scripts is:

{
  "auth": "npx google-artifactregistry-auth",
  "preinstall": "npm run auth",
  "start": "node build/src/index.js",
  "build": "tsc --build",
  "gcp-build": "npm run build",
  "lint": "eslint . --ext .ts",
  "prettier": "prettier --check .",
  "test": "jest",
  "coverage": "jest --coverage --collectCoverageFrom=src/**/*.ts",
  "clean": "rimraf build"
}

[dbburgess]

@kennethye1 This issue has cropped back up for me. I don't see any actual changes, except updating keys in https://github.com/GoogleCloudPlatform/nodejs-docker/pull/264, but is it possible the wrong version was built / tagged when rolling that out?

[kennethye1]

Im looking into it.

[kennethye1]

Should be fixed now.

[dbburgess]

🎉 Thank you for the prompt fix, @kennethye1. You're a hero. 🦸