GoogleCloudPlatform / nodejs-docker

The Node.js Docker image used by Google App Engine Flexible.
https://cloud.google.com/nodejs
Apache License 2.0
405 stars 115 forks source link

Error installing node.js: The Node.js binary could not be verified. #214

Closed jarrodek closed 2 years ago

jarrodek commented 4 years ago

When I run the build I receive the following error:

Sending build context to Docker daemon     75MB
Step 1/15 : FROM launcher.gcr.io/google/nodejs
 ---> d8132d024114
Step 2/15 : RUN install_node v12.16.2
 ---> Running in 80fefff9837f
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 22.0M  100 22.0M    0     0  8969k      0  0:00:02  0:00:02 --:--:-- 8969k
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100  4029  100  4029    0     0  11964      0 --:--:-- --:--:-- --:--:-- 11991
gpg: Signature made Wed Apr 8 19:22:22 2020 UTC using RSA key ID 5599653C gpg: Can't check signature: public key not found
The Node.js binary could not be verified.
This means it may not be an officially released Node.js binary
or may have been tampered with.

Aborting the installation.

The installation can be forced using the --ignore-verification-failure
flag. However, it is strongly recommended that you install a version
of Node.js that can be verified.

Node installation failed: /opt/gcp/runtime/bootstrap_node exited with a non-zero exit code: 1
The command '/bin/sh -c install_node v12.16.2' returned a non-zero code: 1

I have the following configuration:

# Dockerfile extending the generic Node image with application files for a
# single application.
FROM launcher.gcr.io/google/nodejs

# Check to see if the the version included in the base runtime satisfies
# '>=8.12.0', if not then do an npm install of the latest available
# version that satisfies it.
# RUN /usr/local/bin/install_node '>=12.13.0'
RUN install_node v12.16.2

When I do what is suggested it installs the node:

RUN install_node v12.16.2 --ignore-verification-failure

but clearly there is a problem somewhere.

plegner commented 4 years ago

Any update on this? I'm seeing the same error with the latest Google Cloud SDK (309.0.0)

aiimtechdev commented 4 years ago

seems the issue with node 14.10.0. you can try node version 14.9.0 then it works!

prowwid commented 4 years ago

Hello @plegner

in our case, it was the same as for @aiimtechdev.

We had broken deployment to App Engine because it installed Node 14.10.0 as we had "node": "14.x"in the package.json. Seems like the latest release has some issues.

I fixed deploy by changing the engine to a fixed version:

"engines": {
    "node": "14.9"
}

If you are using 12.x - try some of the previous version that worked.

Obviously it does not solve the overall issue with the latest release but at least get things working.

Hope it will help!

plegner commented 4 years ago

Thanks, @prowwid! We were using ">=12.0.0" and had lots of successfully released with that until this week. Fixing the version also worked for us 😄

MatthieuLemoine commented 4 years ago

This error also occurs with the latest release of Node 10 10.23.0 which was released by @richardlau. His signing key is missing in the KEYS file https://github.com/GoogleCloudPlatform/nodejs-docker/blob/master/runtime-image/contents/KEYS, it was added back in july in the Node repo https://github.com/nodejs/node/pull/34397/files

richardlau commented 4 years ago

In addition to my signing key, whoever maintains this repository may also want to add the keys for @ruyadorno (https://github.com/nodejs/node/pull/34628) and @danielleadams (https://github.com/nodejs/node/pull/35545) who have also been recently onboarded as releasers and may sign future releases: https://github.com/nodejs/node/blob/master/README.md#release-keys

donmccasland commented 3 years ago

Taking a look at this.

donmccasland commented 3 years ago

Okay, fixed in staging: onmccasland-macbookpro2:nodejs-docker donmccasland$ docker run -it gcr.io/google-appengine/nodejs@sha256:058266594604d7967a371094f0631c32f93908c37e20c3354d22f1b3a63c9a17 /bin/bash root@d9eee9e2f7c8:/app# install_node v10 % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 20.2M 100 20.2M 0 0 10.6M 0 0:00:01 0:00:01 --:--:-- 10.6M % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 4232 100 4232 0 0 14452 0 --:--:-- --:--:-- --:--:-- 14493 node-v10.23.0-linux-x64.tar.gz: OK Installed Node.js v10.23.0

Starting the release process..

donmccasland commented 3 years ago

Fixed.

elyobo commented 3 years ago

Occurring in GAE flex again with 14.x set.

Step #1: INFO[0047] RUN /usr/local/bin/install_node '14.x'       
Step #1: INFO[0047] cmd: /bin/sh                                 
Step #1: INFO[0047] args: [-c /usr/local/bin/install_node '14.x'] 
Step #1:   % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
Step #1:                                  Dload  Upload   Total   Spent    Left  Speed
100 32.3M  100 32.3M    0     0  43.1M      0 --:--:-- --:--:-- --:--:-- 43.2M
Step #1:   % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
Step #1:                                  Dload  Upload   Total   Spent    Left  Speed
100  3835  100  3835    0     0  24299      0 --:--:-- --:--:-- --:--:-- 24426
Step #1: gpg: Signature made Tue May 11 23:13:29 2021 UTC using RSA key ID 643B6201 gpg: Can't check signature: public key not found
Step #1: The Node.js binary could not be verified.
Step #1: This means it may not be an officially released Node.js binary
Step #1: or may have been tampered with.
Step #1: 
Step #1: Aborting the installation.
Step #1: 
Step #1: The installation can be forced using the --ignore-verification-failure
Step #1: flag. However, it is strongly recommended that you install a version
Step #1: of Node.js that can be verified.
Step #1: 
Step #1: Node installation failed: /opt/gcp/runtime/bootstrap_node exited with a non-zero exit code: 1
Step #1: error building image: error building stage: waiting for process to exit: exit status 1
Finished Step #1
ERROR
ERROR: build step 1 "gcr.io/kaniko-project/executor@sha256:f87c11770a4d3ed33436508d206c584812cd656e6ed08eda1cff5c1ee44f5870" failed: step exited with non-zero status: 1

Edit: setting the package engine version to the last successfully deployed version (14.6.1) instead of 14.x works around the issue for now.

alexnault commented 3 years ago

I confirm: Node 14.17.0 is having that same issue. Note: Node 14.16.0 works fine.

cewald commented 3 years ago

I've the same problem using Google App Engine Flex & Standard: if I downgrade to 14.16.x using "engines": { "node": "14.16.x" } in my package.json I'd work but if I deploy the package in another environment using the standard environment it would again result in an error-message because of the wrong version:

The 'package.json' configuration file must specify a supported nodejs14 version that is compatible with the runtime specified in the deployment. Please pin your application to a compatible major version of the runtime via the 'engines.node' property. Supported 'engines.node' values for the runtime 'nodejs14' are: [14.x.x, 14.X.X, 14.x, 14.X, 14.*.*, 14.*, ~14, ~14.0, ~14.0.0, 14, ^14, ^14.0, ^14.0.0, >=14, >=14.0, >=14.0.0, >14.0, >14.0.0, >=1, >=1.0, >=1.0.0, >1, >1.0, >1.0.0, >=2, >=2.0, >=2.0.0, >2, >2.0, >2.0.0, >=3, >=3.0, >=3.0.0, >3, >3.0, >3.0.0, >=4, >=4.0, >=4.0.0, >4, >4.0, >4.0.0, >=5, >=5.0, >=5.0.0, >5, >5.0, >5.0.0, >=6, >=6.0, >=6.0.0, >6, >6.0, >6.0.0, >=7, >=7.0, >=7.0.0, >7, >7.0, >7.0.0, >=8, >=8.0, >=8.0.0, >8, >8.0, >8.0.0, >=9, >=9.0, >=9.0.0, >9, >9.0, >9.0.0, >=10, >=10.0, >=10.0.0, >10, >10.0, >10.0.0, >=11, >=11.0, >=11.0.0, >11, >11.0, >11.0.0, >=12, >=12.0, >=12.0.0, >12, >12.0, >12.0.0, >=13, >=13.0, >=13.0.0, >13, >13.0, >13.0.0].

So this should be updated in both environments.

pete-beresford-bzy commented 3 years ago

Seems even v14.16 now has key issues (expired):

gpg: Signature made 02/23/21 13:00:40 GMT Standard Time using RSA key ID A1AB005C
gpg: Good signature from "Beth Griggs <bgriggs@redhat.com>" [expired]
gpg:                 aka "Beth Griggs <Bethany.Griggs@uk.ibm.com>" [expired]
gpg: Note: This key has expired!
Primary key fingerprint: 4ED7 78F5 39E3 634C 779C  87C6 D706 2848 A1AB 005C
BethGriggs commented 3 years ago

@pete-beresford-bzy, Hmm...I extended my key back in March but it seems it hasn't properly propagated/synced with the servers - I'll try resyncing/resubmitting.

plegner commented 3 years ago

Why can't AppEngine just pick the last supported version of Node if a range like 14.x is specified in package.json?

Even if we manually specify a supported version like 14.16, the engines key is automatically updated by scripts like Renovate, and it is quite annoying if that leads to unpredictable deployment failures…

danielleadams commented 3 years ago

Are there still issues with 14.7.0? Like @BethGriggs my key was synced a few months ago (Back in January, it looks like https://github.com/nodejs/node/pull/36793), but I've just recently resubmitted/resynched it again to confirm.

brunogabuzomeu commented 3 years ago

Same issue with version 16.11.0

chiefy commented 3 years ago

Same as what @brunogabuzomeu said, we tempoarily solved it by using =16.10.0 which still works.

guavadevelopment commented 3 years ago

FYI 14.18.1 was released 12-Oct-2021 and is having this issue, 14.18.0 can be set manually to deploy

BethGriggs commented 3 years ago

The issue is possibly that the KEYS file in this repository has not been updated to contain @danielleadams's new key (updated in January, https://github.com/nodejs/node/pull/36793).

The Node.js project has recently started to maintain https://github.com/nodejs/release-keys/tree/main/keys as a source of truth for our release keys, which might be a useful resource for keeping this repository in sync with Node.js key updates.

vvision commented 3 years ago

Same issue here, GAE Flex with "engines": { "node": "14.x"},. Was working on monday (2021-10-11). I guess KEYS file need to be updated again. I will try downgrading to unblock deployment, but I'm definitvely not a fan of this solution. Edit: Downgrading to "engines": {"node": "14.18.0"}, fixed it.

Could KEY file updates be automated to prevent such issue to periodically happen ?

chiefy commented 3 years ago

@vvision see @BethGriggs comment - yes it could be automated but I just tried forking + running the tests (they are very broken) and this repo appears unmaintained 👎

@donmccasland any help here please?

wokkaflokka commented 3 years ago

Setting 14.18.0 works as a short term solution -- unfortunately, if you are deploying, say, cloud functions from the same project, "engines": {"node": "14.18.0"} is invalid according to a separate Google technology platform.

Error: package.json in functions directory has an engines field which is unsupported. Valid choices are: {"node": 10|12|14|16}
takayukioda commented 3 years ago

I had same issue for node12.22.7 today.

I have specified node: ^12.22.1 in package.json and failed. Next I ran with node: 12.22.7 and it failed. Finally I ran with node: 12.22.6 and it worked.

according to the nodejs releases, 12.22.7 has been released at 2021/10/12, so I guess that's where it got broken.

BethGriggs commented 3 years ago

according to the nodejs releases, 12.22.7 has been released at 2021/10/12, so I guess that's where it got broken.

Any releases signed by @danielleadams's key will not verify until the keys are updated in this repository, which include the latest security releases released on 2021-10-12 (v12.22.7, v14.18.1, and v16.11.1).

louisgv commented 3 years ago

I checked the key list, it seems Danielle's key is in: https://github.com/GoogleCloudPlatform/nodejs-docker/blob/master/runtime-image/bin/gen_keys.sh#L23

https://github.com/nodejs/node#release-keys

louisgv commented 3 years ago

Oops, her key changed in 14.x branch:

https://github.com/nodejs/node/tree/v14.x-staging#release-keys

1C050899334244A8AF75E53792EF661D867B9DFA -> 74F12602B6F1C4E913FAA37AD3A89613643B6201

takayukioda commented 3 years ago

So I've made a commit to update the GPG key here. https://github.com/takayukioda/nodejs-docker/commit/d9cd69b336d98c4e91d6eaa5ef17039f8eb886ce

According to the contributing guideline I need to propose the change before making PR. Who should I mention to?

Also, I'm not sure how to test and haven't tested yet, so giving me the way to test would be helpful.

chiefy commented 3 years ago

So I've made a commit to update the GPG key here. takayukioda@d9cd69b

According to the contributing guideline I need to propose the change before making PR. Who should I mention to?

Also, I'm not sure how to test and haven't tested yet, so giving me the way to test would be helpful.

https://github.com/GoogleCloudPlatform/nodejs-docker/blob/master/run_test.sh

takayukioda commented 3 years ago

@chiefy ah, thanks a lot! I haven't check repository close enough 🤦🏽‍♂️ I'll keep in my mind in case we have next update.

takayukioda commented 3 years ago

@donmccasland I haven't seen any updates on https://github.com/GoogleCloudPlatform/nodejs-docker/pull/225. Is there anything that I can help?

OleksiiKachan commented 2 years ago

works for me now with 14.18.2. I think this issue is good to close

elyobo commented 2 years ago

It should probably remain open until ongoing fix is found. On Fri, 3 Dec 2021 at 12:50, Oleksii Kachan @.***> wrote:

works for me now with 14.18.2. I think this issue is good to close

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/GoogleCloudPlatform/nodejs-docker/issues/214#issuecomment-985147480, or unsubscribe https://github.com/notifications/unsubscribe-auth/AADDR7UZ5ZJQ5BNUBPEHMPDUPAO4VANCNFSM4MI6GDGA .

geiszla commented 2 years ago

Specifying "engines": { "node": "14.18.0" } still throws the error for me on GAE Flex during build:

Step #1: Step #1: error @bindr/dev@1.0.0: The engine "node" is incompatible with this module. Expected version "14.18.0". Got "12.19.0"

Is this fix supposed to fix the build environment as well?

takayukioda commented 2 years ago

@geiszla your issue seems irrelevant to this issue. The fix mentioned in above is about using latest version fails due to GPG key mismatch.

Your issue is about mismatch of runtime node version and engine version specified in package.json.

niciliketo commented 2 years ago

I am sseing a similar issue for 16.4.x It seems to have started from 2022-03-26

Step #1: INFO[0022] COPY . /app/                                 
Step #1: INFO[0022] Taking snapshot of files...                  
Step #1: INFO[0023] RUN /usr/local/bin/install_node '>=16.4.x'   
Step #1: INFO[0023] cmd: /bin/sh                                 
Step #1: INFO[0023] args: [-c /usr/local/bin/install_node '>=16.4.x'] 
Step #1:   % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
Step #1:                                  Dload  Upload   Total   Spent    Left  Speed
100 40.9M  100 40.9M    0     0   102M      0 --:--:-- --:--:-- --:--:--  102M
Step #1:   % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
Step #1:                                  Dload  Upload   Total   Spent    Left  Speed
100  4009  100  4009    0     0  41326      0 --:--:-- --:--:-- --:--:-- 41760
Step #1: gpg: Signature made Tue Mar 22 14:38:58 2022 UTC using RSA key ID E57C7D57 gpg: Can't check signature: public key not found
Step #1: The Node.js binary could not be verified.
Step #1: This means it may not be an officially released Node.js binary
Step #1: or may have been tampered with.
Step #1: 
Step #1: Aborting the installation.
Step #1: 
Step #1: The installation can be forced using the --ignore-verification-failure
Step #1: flag. However, it is strongly recommended that you install a version
Step #1: of Node.js that can be verified.
niciliketo commented 2 years ago

am sseing a similar issue for 16.4.x

@chiefy apologies, I should have provided more context. I have raised an issue with GCP, who have managed to reproduce it. https://issuetracker.google.com/issues/227382491 As it was a similar error to the one reported here, and this is a GCP repo, I had assumed this would be an appropriate place to report it. I have since learnt that there are many GCP nodejs repos, and I am not sure this is the one used by the AppEngine flex environment that I am using.

chiefy commented 2 years ago

@niciliketo the 😕 was for the maintainers, sorry. Disappointing this hasn't been taken care of yet?

jinglundong commented 2 years ago

FYI, fix is almost ready. We plan to release it to prod tomorrow 4/5/2022.

OPNutrition commented 2 years ago

@jinglundong update?

jinglundong commented 2 years ago

I verified this fix in prod with "16.4.x" in package.json and "runtime: node" in app.yaml. This should have been fixed. Please let me know if this is still an issue for you.

We are going to add end to end tests that execute this fuzzy version behavior. Hope we detect this type of issue faster next time.

ishak-lm commented 2 years ago

@jinglundong Still not working on our side

OPNutrition commented 2 years ago

@jinglundong thanks for the update.

I tried using 14.x and 14.18.x and neither worked.

It did pass the initial install with 16.13.x as you had verified

Unfortunately one of the main packages in our dependencies does not support v16

jinglundong commented 2 years ago

@ishak-lm, @OPNutrition

Sorry to hear that. Could you share instructions to reproduce the problem?

OPNutrition commented 2 years ago

@jinglundong here is my config

package.json:

  "engines": {
    "node": "14.x",
    "npm": ">=6.0.0"
  }

app.yaml:

runtime: nodejs

env: flex

manual_scaling:
  instances: 1

resources:
  cpu: 1
  memory_gb: 0.5
  disk_size_gb: 10

error:

Step #1: [1/5] Validating package.json...
Step #1: error onpoint-nutrition@0.1.0: The engine "node" is incompatible with this module. Expected version "14.x". Got "12.19.0"
jinglundong commented 2 years ago

I couldn't reproduce the error with

  "engines": {
    "node": "14.x",
    "npm": ">=6.0.0"
  }

When I deploy, the version picked were: Installed Node.js v14.19.1 and Installed "npm@8.6.0" with binaries. Could you try this combination. Another idea is to use node >= 14.x.

ishak-lm commented 2 years ago

@jinglundong I have almost the same configuration:

  "engines": {
    "node": "14.x",
    "npm": ">=6.0.0"
  }
runtime: nodejs

env: flex

# Resources configuration (for each server)
resources:
  cpu: 1
  memory_gb: 2
  disk_size_gb: 10

# Configuration of automatic scaling
# block max_num_instance (3)
automatic_scaling:
  min_num_instances: 1
  max_num_instances: 1
  cool_down_period_sec: 180
  cpu_utilization:
    target_utilization: 0.6
# [END app_yaml]
Step #1: yarn install v1.22.18
Step #1: [1/5] Validating package.json...
Step #1: error *************@1.5.30: The engine "node" is incompatible with this module. Expected version "14.x". Got "12.19.0"
Step #1: error Found incompatible module.
Step #1: info Visit https://yarnpkg.com/en/docs/cli/install for documentation about this command.
Step #1: error building image: error building stage: waiting for process to exit: exit status 1
Finished Step #1
gcloud app deploy app-dev.yaml --project=*******
jinglundong commented 2 years ago

@ishak-lm I think this is a different issue. "The engine "node" is incompatible with this module. Expected version "14.x". Got "12.19.0" is not complaining about the pgp key. Could you file another issue and try the specific version that I suggested? node: 14.19.1 and npm: 8.6.0?

I'm going to close this issue for now. If anyone's still seeing Can't check signature: public key not found The Node.js binary could not be verified. please reopen.

chiefy commented 2 years ago

Just to note, this is going to have to be re-opened the next time a new GPG key is added, the problem wasn't solved.

jinglundong commented 2 years ago

That is correct. I'm listing a few ideas below following my preferred order, but I haven't had a chance to figure out the details. Please advice if anyone has better solutions.

  1. We have a plan to simplify how we build Flex applications. I believe it's going to solve this problem, or at least be able to patch it much faster. This is the actual fix, but it will likely take around two quarters to develop.
  2. If new keys are added and announced before they are being used (which would cause the GPG check failure), we are happy to monitor that list of keys more closely. This is a short term solution.
  3. We may have a way to detach from latest, and pin to a relatively new node version. Not always using the latest version of node may cause problems for some users too.