search

node-red / node-red-docker

Repository for all things Node-RED and Docker related
Apache License 2.0
481 stars 383 forks source link

[err] npm WARN read-shrinkwrap This version of npm is compatible with lockfileVersion@1 #309

Closed bastelbert closed 2 years ago

bastelbert commented 2 years ago

What are the steps to reproduce?

Install new node or update an existing one.

What happens?

No installation or update possible. Error messages:

[err] npm WARN read-shrinkwrap This version of npm is compatible with lockfileVersion@1, but package-lock.json was generated for lockfileVersion@2. I'll try to do my best with it!

2022-03-19T22:06:59.945Z [err] ENOENT: no such file or directory, open '/data/node_modules/.staging/node-red-contrib-web-worldmap-2af8d78c/node_modules/cgi/package.json' 2022-03-19T22:06:59.946Z [err] npm WARN tar 2022-03-19T22:06:59.946Z [err] ENOENT: no such file or directory, open '/data/node_modules/.staging/es-abstract-9b694a75/2017/TimeFromYear.js'

What do you expect to happen?

Node installation or update should work.

Please tell us about your environment:

[ ] Platform/OS: Raspbian Bullseye

[ ] Browser: FLOWS | flows.json NODE_PATH | /usr/src/node-red/node_modules:/data/node_modules NODE_RED_VERSION | v2.2.2 NODE_VERSION | 14.18.2 PATH | /usr/src/node-red/node_modules/.bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin TZ | Europe/Berlin YARN_VERSION | 1.22.15

hardillb commented 2 years ago

Just to be 100% clear, which container did you start with? I'm guessing nodered/node-red:latest-14?

Any particular reason you are not using the NodeJS 16 based container (nodered/node-red:latest)?

Also which node did you try and install and how? Was it via the palette manager?

bastelbert commented 2 years ago

Thanks for your fast reply! I'm using "image: nodered/node-red:latest" in docker-compose.yml.

I've tried to update and install several different nodes via the palette manager, always the same error message.

hardillb commented 2 years ago

Sorry, my mistake, latest is still 14 as that was what was still LTS for 2.2.0 release.

Please answer the question as to which nodes you tried to install, it is important.

hardillb commented 2 years ago

And if you have a volume mounted to the /data directory

bastelbert commented 2 years ago

e.g. update of node-red-contrib-deconz to 2.3.3 or update of node-red-contrib-mqtt-broker to 0.2.9:

2022-03-19T22:37:09.937Z Installieren : node-red-contrib-mqtt-broker 0.2.9 2022-03-19T22:37:10.050Z npm install --no-audit --no-update-notifier --no-fund --save --save-prefix=~ --production --engine-strict node-red-contrib-mqtt-broker@0.2.9 2022-03-19T22:37:14.852Z [err] npm 2022-03-19T22:37:14.852Z [err] WARN deprecated node-red-contrib-mqtt-broker@0.2.9: deprecated 2022-03-19T22:37:15.816Z [err] npm WARN read-shrinkwrap This version of npm is compatible with lockfileVersion@1, but package-lock.json was generated for lockfileVersion@2. I'll try to do my best with it! 2022-03-19T22:37:19.319Z [err] npm 2022-03-19T22:37:19.320Z [err] WARN deprecated uuid@2.0.3: Please upgrade to version 7 or higher. Older versions may use Math.random() in certain circumstances, which is known to be problematic. See https://v8.dev/blog/math-random for details. 2022-03-19T22:37:19.607Z [err] npm WARN 2022-03-19T22:37:19.607Z [err] deprecated mongodb@2.1.21: Please upgrade to 2.2.19 or higher 2022-03-19T22:37:20.295Z [err] npm WARN deprecated node-uuid@1.4.8: Use uuid module instead 2022-03-19T22:37:23.521Z [err] npm 2022-03-19T22:37:23.521Z [err] WARN deprecated tar@2.2.2: This version of tar is no longer supported, and will not receive security updates. Please upgrade asap. 2022-03-19T22:37:26.606Z [err] npm WARN

Entry in docker-compose.yml: volumes:

-> /root/compose/nodered/data /data

hardillb commented 2 years ago

This will be a problem with one of the nodes you have installed. Best bet is to attach the /data/package.json file so we can see all the node you have installed.

bastelbert commented 2 years ago

cat package.json

{
  "name": "node-red-project",
  "description": "A Node-RED Project",
  "version": "0.0.1",
  "private": true,
  "dependencies": {
    "node-red-contrib-controltimer": "~0.1.0",
    "node-red-contrib-deconz": "~1.3.4",
    "node-red-contrib-homebridge-automation": "~0.0.79",
    "node-red-contrib-influxdb": "~0.6.1",
    "node-red-contrib-loxone": "~0.10.11",
    "node-red-contrib-luxtronik2": "~0.4.0",
    "node-red-contrib-mqtt-broker": "~0.2.5",
    "node-red-contrib-msg-resend": "~1.0.0",
    "node-red-contrib-onvif": "~1.0.3",
    "node-red-contrib-simpletime": "~2.10.0",
    "node-red-contrib-solaredge": "~0.1.0",
    "node-red-contrib-web-worldmap": "~2.16.2",
    "node-red-dashboard": "~2.29.3"
  }
}
hardillb commented 2 years ago

All of those nodes install cleanly x86_64 container.

Are you running on 32 or 64bit Raspbain?

Also are you sure that the volume you have mounted hasn't ever run out of disk space? The only time /data/node_modules/.staging is created is when unpacking new npm modules and if there is a disk space or permissions problems then it will fail.

bastelbert commented 2 years ago

I think it's 32bit Raspbian: uname -m armv7l

I'm not quite sure but I think the Raspi ran out of disk space once a while ago. The directory is empty now: ~/compose/nodered/data/node_modules/.staging# ll drwxr-xr-x 2 pi pi 4096 19. Mär 23:38 . drwxrwxrwx 498 pi pi 20480 19. Mär 23:38 ..

bastelbert commented 2 years ago

Do you see any chance to solve this issue without removing all nodes as described with this workaround? https://stackoverflow.com/questions/44573592/why-do-node-modules-go-into-staging-folder

hardillb commented 2 years ago

I suggest you back up the following files

With these you should be able to recreate your flows if needed.

Then run the following commands in the container (use docker exec -it <container name> /bin/sh to get a shell in the container.

Then restart the container.

bastelbert commented 2 years ago

Thank you. But that means I have to reinstall every node, right?

hardillb commented 2 years ago

The npm install will use the package.json to reinstall all the nodes in one go.

bastelbert commented 2 years ago

Hi, I did it like you said and node-red came up again afterwards, but I still get the same errors when updating or installing nodes -> ENOENT: no such file or directory, lstat '/data/node_modules/.staging/node-red-contrib-web-worldmap-2af8d78c/.github'

~ $ rm package-lock.json ~ $ rm -rf node_modules ~ $ npm install

bcrypt@5.0.1 install /usr/src/node-red/node_modules/bcrypt node-pre-gyp install --fallback-to-build

node-pre-gyp ERR! install response status 404 Not Found on https://github.com/kelektiv/node.bcrypt.js/releases/download/v5.0.1/bcrypt_lib-v5.0.1-napi-v3-linux-arm-musl.tar.gz node-pre-gyp WARN Pre-built binaries not installable for bcrypt@5.0.1 and node@14.18.2 (node-v83 ABI, musl) (falling back to source compile with node-gyp) node-pre-gyp WARN Hit error response status 404 Not Found on https://github.com/kelektiv/node.bcrypt.js/releases/download/v5.0.1/bcrypt_lib-v5.0.1-napi-v3-linux-arm-musl.tar.gz make: Entering directory '/usr/src/node-red/node_modules/bcrypt/build' CC(target) Release/obj.target/nothing/../node-addon-api/nothing.o AR(target) Release/obj.target/../node-addon-api/nothing.a COPY Release/nothing.a CXX(target) Release/obj.target/bcrypt_lib/src/blowfish.o CXX(target) Release/obj.target/bcrypt_lib/src/bcrypt.o CXX(target) Release/obj.target/bcrypt_lib/src/bcrypt_node.o SOLINK_MODULE(target) Release/obj.target/bcrypt_lib.node COPY Release/bcrypt_lib.node COPY /usr/src/node-red/node_modules/bcrypt/lib/binding/napi-v3/bcrypt_lib.node TOUCH Release/obj.target/action_after_build.stamp make: Leaving directory '/usr/src/node-red/node_modules/bcrypt/build' npm notice created a lockfile as package-lock.json. You should commit this file. added 290 packages from 374 contributors and audited 290 packages in 31.286s

31 packages are looking for funding run npm fund for details

found 0 vulnerabilities

hardillb commented 2 years ago

Just to be 100% clear, where and how did you run those commands? because it looks like you ran them in the /usr/src/node-red and NOT in the /data directory as asked.

It looks like you missed the first instruction off the list.

bastelbert commented 2 years ago

THANK YOU! Sorry, you're right, I did not change to /data I ran the commands again, now the .staging errors don't come up anymore. Unfortunately one node is broken now, I cannot reinstall node-red-contrib-onvif anymore.

sharp@0.28.3 install /data/node_modules/sharp 2022-03-20T17:22:11.056Z [out] > (node install/libvips && node install/dll-copy && prebuild-install) || (node install/can-compile && node-gyp rebuild && node install/dll-copy) 2022-03-20T17:22:11.056Z [out] 2022-03-20T17:22:11.531Z [out] sharp: Downloading https://github.com/lovell/sharp-libvips/releases/download/v8.10.6/libvips-8.10.6-linuxmusl-armv7.tar.br 2022-03-20T17:22:11.791Z [err] sharp: Installation error: Prebuilt libvips 8.10.6 binaries are not yet available for linuxmusl-armv7 2022-03-20T17:22:11.792Z [out] sharp: Please see https://sharp.pixelplumbing.com/install for required dependencies

hardillb commented 2 years ago

You'll have to raise that with the owner of that node, my guess is you've upgraded at some point and that node has not been updated to support NodeJS 14

dceejay commented 2 years ago

I would suspect that is now more to do with the fact the onvif node is using native code bindings which may not exist in the same way for the Alpine based OS we use for docker. You may need to build a custom container based off of debian.

bastelbert commented 2 years ago

Has anything changed in your Docker image recently? I had already installed and actively used the Onvif node in this image. It was only after I had carried out the clean-up described above that the node could no longer be installed.

bastelbert commented 2 years ago

Seems like I really broke something with the clean-up described above. I cannot uninstall (unused) nodes anymore, every time issues with sharp:

2022-03-21T08:55:08.930Z Entfernen : node-red-contrib-mqtt-broker

2022-03-21T08:55:10.567Z npm remove --no-audit --no-update-notifier --no-fund --save node-red-contrib-mqtt-broker 2022-03-21T08:55:28.318Z [out] 2022-03-21T08:55:28.318Z [out] > sharp@0.28.3 install /data/node_modules/sharp 2022-03-21T08:55:28.318Z [out] > (node install/libvips && node install/dll-copy && prebuild-install) || (node install/can-compile && node-gyp rebuild && node install/dll-copy) 2022-03-21T08:55:28.318Z [out] 2022-03-21T08:55:28.682Z [out] sharp: Downloading https://github.com/lovell/sharp-libvips/releases/download/v8.10.6/libvips-8.10.6-linuxmusl-armv7.tar.br 2022-03-21T08:55:28.961Z [err] sharp: Installation error: Prebuilt libvips 8.10.6 binaries are not yet available for linuxmusl-armv7 2022-03-21T08:55:28.961Z [out] sharp: Please see https://sharp.pixelplumbing.com/install for required dependencies 2022-03-21T08:55:29.896Z [err] npm 2022-03-21T08:55:29.896Z [err] ERR! code ELIFECYCLE 2022-03-21T08:55:29.896Z [err] npm ERR! errno 1 2022-03-21T08:55:29.933Z [err] npm ERR! 2022-03-21T08:55:29.933Z [err] sharp@0.28.3 install: (node install/libvips && node install/dll-copy && prebuild-install) || (node install/can-compile && node-gyp rebuild && node install/dll-copy) 2022-03-21T08:55:29.933Z [err] npm ERR! Exit status 1 2022-03-21T08:55:29.933Z [err] npm ERR! 2022-03-21T08:55:29.933Z [err] npm ERR! 2022-03-21T08:55:29.933Z [err] Failed at the sharp@0.28.3 install script. 2022-03-21T08:55:29.933Z [err] npm ERR! This is probably not a problem with npm. There is likely additional logging output above. 2022-03-21T08:55:29.984Z [err] 2022-03-21T08:55:29.984Z [err] npm ERR! A complete log of this run can be found in: 2022-03-21T08:55:29.984Z [err] npm ERR! /data/.npm/_logs/2022-03-21T08_55_29_958Z-debug.log 2022-03-21T08:55:30.004Z rc=1

hardillb commented 2 years ago

Please take some time to read the errors that are shown, this npm still trying to instal libvips which is a dependency of the npm module sharp which is required for the node-red-contrib-onvif node.

It will try and install that for every npm command you run until you uninstall it because it's in the package.json file.

At this point this issue is closed, you will need to talk to the owner of node-red-contrib-onvif (or more likely sharp) about support for installing on Alpine linux (with muscl instead of glibc) or move to building your own debian based image (there are instructions for this in the repo)

The link given in the error (https://sharp.pixelplumbing.com/install) says it should support Alpine and Muscl so you will need to talk them about why it's not working here.

hardillb commented 2 years ago

I have raised the issue against the sharp project for you https://github.com/lovell/sharp/issues/3150, it will be up to you to follow up with them and supply any more information they may require.

bastelbert commented 2 years ago

Thank you very much for your great support, this is awesome! Nevertheless it seems like my nodered installation is totally broken. I can update and install new nodes (most of them, unfortunately not the onvif one), but as soon I try to uninstall a node, I get the error messages regarding sharp. Example: uninstallation of node-red-node-email

2022-03-21T11:04:36.966Z [err] make: *** [fs-ext.target.mk:109: Release/obj.target/fs-ext/fs-ext.o] Error 1 2022-03-21T11:04:36.967Z [out] make: Leaving directory '/data/node_modules/fs-ext/build' 2022-03-21T11:04:36.969Z [err] gyp 2022-03-21T11:04:36.969Z [err] ERR! 2022-03-21T11:04:36.970Z [err] build error 2022-03-21T11:04:36.971Z [err] gyp 2022-03-21T11:04:36.971Z [err] ERR! stack Error: make failed with exit code: 2 2022-03-21T11:04:36.971Z [err] gyp 2022-03-21T11:04:36.971Z [err] ERR! stack at ChildProcess.onExit (/usr/local/lib/node_modules/npm/node_modules/node-gyp/lib/build.js:194:23) 2022-03-21T11:04:36.971Z [err] gyp 2022-03-21T11:04:36.971Z [err] ERR! 2022-03-21T11:04:36.971Z [err] stack at ChildProcess.emit (events.js:400:28) 2022-03-21T11:04:36.971Z [err] gyp 2022-03-21T11:04:36.971Z [err] ERR! stack at Process.ChildProcess._handle.onexit (internal/child_process.js:282:12) 2022-03-21T11:04:36.972Z [err] gyp ERR! 2022-03-21T11:04:36.972Z [err] System Linux 5.10.103-v7l+ 2022-03-21T11:04:36.972Z [err] gyp ERR! command "/usr/local/bin/node" "/usr/local/lib/node_modules/npm/node_modules/node-gyp/bin/node-gyp.js" "configure" "build" 2022-03-21T11:04:36.972Z [err] gyp 2022-03-21T11:04:36.973Z [err] ERR! cwd /data/node_modules/fs-ext 2022-03-21T11:04:36.973Z [err] gyp ERR! 2022-03-21T11:04:36.973Z [err] node -v v14.18.2 2022-03-21T11:04:36.973Z [err] gyp 2022-03-21T11:04:36.973Z [err] ERR! node-gyp -v v5.1.0 2022-03-21T11:04:36.973Z [err] gyp 2022-03-21T11:04:36.973Z [err] ERR! not ok 2022-03-21T11:04:37.005Z [out] 2022-03-21T11:04:37.005Z [out] > sharp@0.28.3 install /data/node_modules/sharp 2022-03-21T11:04:37.005Z [out] > (node install/libvips && node install/dll-copy && prebuild-install) || (node install/can-compile && node-gyp rebuild && node install/dll-copy) 2022-03-21T11:04:37.005Z [out] 2022-03-21T11:04:37.486Z [out] sharp: Downloading https://github.com/lovell/sharp-libvips/releases/download/v8.10.6/libvips-8.10.6-linuxmusl-armv7.tar.br 2022-03-21T11:04:37.766Z [err] sharp: Installation error: Prebuilt libvips 8.10.6 binaries are not yet available for linuxmusl-armv7 2022-03-21T11:04:37.766Z [out] sharp: Please see https://sharp.pixelplumbing.com/install for required dependencies

hardillb commented 2 years ago

I've already answered that, you need to remove node-red-contrib-onvif from the /data/package.json file. I will not respond again on this closed issue.