npm can't install or update

columbian-chris commented 3 years ago

I have tried everything I could find and nothing will fix my problem. I cannot for the life of me get this npm install or update error to resolve.

npm ERR! code ENOENT
npm ERR! syscall open
npm ERR! path /srv/www/columbian/public_html/wp-content/themes/columbian2/node_modules/ansi-align/node_modules/ansi-regex/package.json.4193727334
npm ERR! errno -2
npm ERR! enoent ENOENT: no such file or directory, open '/srv/www/columbian/public_html/wp-content/themes/columbian2/node_modules/ansi-align/node_modules/ansi-regex/package.json.4193727334'
npm ERR! enoent This is related to npm not being able to find a file.

It is not specific to this package. Trying different things throws the same error and gets caught up on a different package after I try different things.

I have tried:

clearing the cache (npm cache clean --force)
deleting the node_modules directory and doing a fresh npm install
tried using Node 12 and 10
tried running as root user
tried chmod-ing 777 to everything in the node_modules dir
deleting the package-lock.json file and trying again
manually deleting any hidden/temporary folders in node_modules created during attemtps
using some custom npm commands some people suggested on SO:
- npm rebuild
- npm install --cache /tmp/empty-cache
- npm cache verify
- npm prune
- npm update --no-bin-links

...nothing seems to work. Any suggestions?

Using: MacOS 11.5.2 VVV 3.7.2 Vagrant 2.2.18 virtualbox 6.1.26

welcome[bot] commented 3 years ago

Thanks for opening your first issue here! Be sure to follow the issue template and include your OS/Vagrant/VVV versions! Don't forget you can get support in the VVV slack at https://varyingvagrantvagrants.org/docs/en-US/slack/

VVV
VVV Slack Workspace
Join the VVV Slack Workspace

tomjn commented 3 years ago

It's extremely unlikely that there is a VVV oriented solution to this. The only things I can suggest are all Windows related issues regarding symlink support, which is not an issue on MacOS hosts.

tomjn commented 3 years ago

If I had to grasp at straws:

check that your guest and your host haven't ran out of disk space
try this in a folder that is not mounted e.g. /tmp
does it work if you run it on the host? VVV may serve the files and database but nothing prevents you using npm/node on MacOS
booo 6.1.26 booooo, terrible version, boooo, burn with the fire of a thousand suns, boooo, I have lots of anecdotes about this version of VirtualBox, all of them are universally bad for different ambiguous reasons. All anecdotes about downgrading are about how they fixed mysterious issues.
also, VVV is not npm, I can't do much to help if I can't reproduce the issue

columbian-chris commented 3 years ago

I spun up a fresh default VVV installation without any customization or configuration and provisioned the vagrant box and ssh'ed into it and I experienced the same issue in it. Also because it's a completely fresh install, that implies to me that there isn't a disk space issue happening.

I ran npm install in the /tmp dir and it successfully ran, so what can we learn from that ???

I was able to successfully use npm without error on the host (MacOS).

I tried to downgrade VirtualBox to 6.0 and the installation would just fail, but duly noted.

tomjn commented 3 years ago

@columbian-chris do you have instructions to reproduce? Where is gulp-boilerplate coming from?

I tried to downgrade VirtualBox to 6.0 and the installation would just fail, but duly noted.

kernel extensions need to be removed, you have to run the Virtualbox uninstaller, you can' just install an older version over the top

tomjn commented 3 years ago

I'm sure at some point the command you were running that was failing was in this issue? You're describing the results of what you're trying to do but you've never actually shared what it is that you did, please share explicit instructions with commands and links so that I can reproduce step for step what you did locally at my end

tomjn commented 3 years ago

also stable branchordevelop` branch?

columbian-chris commented 3 years ago

I'm sure at some point the command you were running that was failing was in this issue? You're describing the results of what you're trying to do but you've never actually shared what it is that you did, please share explicit instructions with commands and links so that I can reproduce step for step what you did locally at my end

The easiest reproduction could possibly be to run an npm command with the list of packages I was trying to install so I'll give you that:

npm install bootstrap bootstrap-icons gulp gulp-autoprefixer gulp-cli gulp-concat gulp-csso gulp-imagemin gulp-livereload gulp-plumber gulp-sass gulp-sourcemaps gulp-uglify intersection-observer js-cookie popper.js slim-select tablesorter tiny-date-picker

columbian-chris commented 3 years ago

also stable branchordevelop` branch?

Sorry, of which software? I didn't use the development branch or release of anything I was using. It should all be stable or LTS.

columbian-chris commented 3 years ago

kernel extensions need to be removed, you have to run the Virtualbox uninstaller, you can' just install an older version over the top

I tried to use the uninstall tool, but MacOS prevented me from doing so and said “VirtualBox_Uninstall.tool” cannot be opened because it is from an unidentified developer.

tomjn commented 3 years ago

Sorry, of which software? I didn't use the development branch or release of anything I was using. It should all be stable or LTS.

@columbian-chris of VVV, we don't have an LTS release

I tried to use the uninstall tool, but MacOS prevented me from doing so and said “VirtualBox_Uninstall.tool” cannot be opened because it is from an unidentified developer.

Go to the security options pane of system preferences and press allow then try again.

tomjn commented 3 years ago

Also that's a lot of packages to install at the same time!

Here's my splash screen for reference:

❯ vagrant up --provision
__ __ __ __
\ V\ V\ V / v3.7.2 Path:"/Users/tarendai/dev/vvv"
 \_/\_/\_/  git::develop(4dcce2f)

Platform: darwin19 shell:/bin/bash vagrant-goodhosts vagrant-vbguest shared_db_folder_disabled
Vagrant: v2.2.18, virtualbox: v6.1.24

Docs:       https://varyingvagrantvagrants.org/
Contribute: https://github.com/varying-vagrant-vagrants/vvv
Dashboard:  http://vvv.test

I tried to run the command and it succeeded when ran in an empty folder:

vagrant@vvv:/srv/www/wordpress-one/public_html/wp-content/themes$ mkdir npmtest
vagrant@vvv:/srv/www/wordpress-one/public_html/wp-content/themes$ cd npmtest/
vagrant@vvv:/srv/www/wordpress-one/public_html/wp-content/themes/npmtest$ npm install bootstrap bootstrap-icons gulp gulp-autoprefixer gulp-cli gulp-concat gulp-csso gulp-imagemin gulp-livereload gulp-plumber gulp-sass gulp-sourcemaps gulp-uglify intersection-observer js-cookie popper.js slim-select tablesorter tiny-date-picker
npm WARN deprecated popper.js@1.16.1: You can find the new Popper v2 at @popperjs/core, this package is dedicated to the legacy v1
npm WARN deprecated chokidar@2.1.8: Chokidar 2 will break on node v14+. Upgrade to chokidar 3 with 15x less dependencies.
npm WARN deprecated urix@0.1.0: Please see https://github.com/lydell/urix#deprecated
npm WARN deprecated resolve-url@0.2.1: https://github.com/lydell/resolve-url#deprecated
npm WARN deprecated fsevents@1.2.13: fsevents 1 will break on node v14+ and could be using insecure binaries. Upgrade to fsevents 2.
npm WARN deprecated uuid@3.4.0: Please upgrade  to version 7 or higher.  Older versions may use Math.random() in certain circumstances, which is known to be problematic.  See https://v8.dev/blog/math-random for details.
npm WARN deprecated core-js@2.6.12: core-js@<3.3 is no longer maintained and not recommended for usage due to the number of issues. Because of the V8 engine whims, feature detection in old core-js versions could cause a slowdown up to 100x even if nothing is polyfilled. Please, upgrade your dependencies to the actual version of core-js.
npm WARN deprecated cross-spawn-async@2.2.5: cross-spawn no longer requires a build toolchain, use it instead

> core-js@2.6.12 postinstall /srv/www/wordpress-one/public_html/wp-content/themes/npmtest/node_modules/core-js
> node -e "try{require('./postinstall')}catch(e){}"

Thank you for using core-js ( https://github.com/zloirock/core-js ) for polyfilling JavaScript standard library!

The project needs your help! Please consider supporting of core-js on Open Collective or Patreon: 
> https://opencollective.com/core-js 
> https://www.patreon.com/zloirock 

Also, the author of core-js ( https://github.com/zloirock ) is looking for a good job -)

> gifsicle@5.2.0 postinstall /srv/www/wordpress-one/public_html/wp-content/themes/npmtest/node_modules/gifsicle
> node lib/install.js

  ✔ gifsicle pre-build test passed successfully

> mozjpeg@7.1.0 postinstall /srv/www/wordpress-one/public_html/wp-content/themes/npmtest/node_modules/mozjpeg
> node lib/install.js

  ✔ mozjpeg pre-build test passed successfully

> optipng-bin@7.0.0 postinstall /srv/www/wordpress-one/public_html/wp-content/themes/npmtest/node_modules/optipng-bin
> node lib/install.js

  ✔ optipng pre-build test passed successfully
npm WARN saveError ENOENT: no such file or directory, open '/srv/www/wordpress-one/public_html/wp-content/themes/npmtest/package.json'
npm notice created a lockfile as package-lock.json. You should commit this file.
npm WARN optional SKIPPING OPTIONAL DEPENDENCY: fsevents@^1.2.7 (node_modules/chokidar/node_modules/fsevents):
npm WARN notsup SKIPPING OPTIONAL DEPENDENCY: Unsupported platform for fsevents@1.2.13: wanted {"os":"darwin","arch":"any"} (current: {"os":"linux","arch":"x64"})
npm WARN enoent ENOENT: no such file or directory, open '/srv/www/wordpress-one/public_html/wp-content/themes/npmtest/package.json'
npm WARN bootstrap@5.1.1 requires a peer of @popperjs/core@^2.10.1 but none is installed. You must install peer dependencies yourself.
npm WARN npmtest No description
npm WARN npmtest No repository field.
npm WARN npmtest No README data
npm WARN npmtest No license field.

+ gulp-plumber@1.2.1
+ gulp-livereload@4.0.2
+ gulp-concat@2.6.1
+ gulp@4.0.2
+ gulp-cli@2.3.0
+ bootstrap-icons@1.5.0
+ bootstrap@5.1.1
+ js-cookie@3.0.1
+ intersection-observer@0.12.0
+ gulp-sourcemaps@3.0.0
+ gulp-uglify@3.0.2
+ popper.js@1.16.1
+ gulp-imagemin@8.0.0
+ gulp-autoprefixer@8.0.0
+ gulp-csso@4.0.1
+ gulp-sass@5.0.0
+ tablesorter@2.31.3
+ slim-select@1.27.0
+ tiny-date-picker@4.0.0-beta6
added 901 packages from 382 contributors and audited 903 packages in 80.225s

62 packages are looking for funding
  run `npm fund` for details

found 8 vulnerabilities (2 low, 2 moderate, 4 high)
  run `npm audit fix` to fix them, or `npm audit` for details

   ╭───────────────────────────────────────────────────────────────╮
   │                                                               │
   │      New major version of npm available! 6.14.10 → 7.5.2      │
   │   Changelog: https://github.com/npm/cli/releases/tag/v7.5.2   │
   │               Run npm install -g npm to update!               │
   │                                                               │
   ╰───────────────────────────────────────────────────────────────╯

vagrant@vvv:/srv/www/wordpress-one/public_html/wp-content/themes/npmtest$

tomjn commented 3 years ago

Node v14.18.0 and npm 6.14.15

columbian-chris commented 3 years ago

Go to the security options pane of system preferences and press allow then try again.

I did so and used the uninstall tool and tried reinstalling and it still fails for some reason. Maybe it's not compatible with my version of MacOS.

columbian-chris commented 3 years ago

If it's working fine on your end, I'm not sure what to think at this point. I feel like I'm going to have to just find a workaround.

tomjn commented 3 years ago

Did you restart your machine? And have you checked the system preferences security panel for new permission checks? virtualbox needs root space kernel extensions it needs them removed and a hard restart, as well as various prompts to allow it to do these things or it will say it failed.

I’d recommend looking up instructions and following them meticulously in the exact order and actions suggested

On Fri, 1 Oct 2021 at 21:38, columbian-chris @.***> wrote:

If it's working fine on your end, I'm not sure what to think at this point. I feel like I'm going to have to just find a workaround.

— You are receiving this because you commented.

Reply to this email directly, view it on GitHub https://github.com/Varying-Vagrant-Vagrants/VVV/issues/2529#issuecomment-932536184, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAAOLZY4UCSAP56BKYMINJTUEYL5VANCNFSM5FC7235Q . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.

columbian-chris commented 3 years ago

I will have to try downgrading virtualbox again when I get a chance with deeper instructions to ensure I'm completely clearing out everything.

Today, I can get a fresh npm install to run if I delete all the node modules and the package-lock.json file. I'm noticing that it always is leaving behind a ton of hidden folders in the node_modules dir and then throwing errors about files within them (like ENOTEMPTY: directory not empty) if I don't delete them before running another npm command.

Example bunch of hidden files left behind:

.acorn-t9RSi9qX
.atob-ddgO3NEi
.autoprefixer-kzS2sdhU
.bin
.depcheck-ej1rzFMr
.esprima-9YzcM93c
.fast-xml-parser-UwCb0HqE
.gifsicle-QTps6ljg
.gulp-cli-6dg0wP4t
.gulp-XB0xevT6
.he-0ImkaW7P
.js-yaml-4FQUlnc2
.jsesc-7WKh6D5K
.mkdirp-jzRFfqsy
.mozjpeg-RYpu1dTP
.nanoid-3feEBV0E
.npm-check-WDGskL0C
.optipng-bin-IZOtOHnj
.package-lock.json
.rc-b7AY4sMo
.sass-7n1Z7SWK
.seek-bzip-LNXBn8lT
.semver-KxQw6ytQ
.svgo-J2Blsq01
.uglify-js-AABh06zL
.uuid-hbdCwbDs
.which-Lifs4sPv

When npm is finished installing a package, I expect it's supposed to remove these. To confirm that, it is working normally when I try inside the /tmp/ directory. I haven't spotted any permissions issues so I'm not sure why npm isn't successfully removing these.

tomjn commented 3 years ago

can you try installing the nvm utility and use it to install Node then update npm? Something like nvm use lts && npm update -g npm? ( untested commands )

As an aside, have you tried running it on the host instead? It's great having node inside the VM particularly for running node projects, but watchers always run best on the host. VMs introduce micro-caches that sometimes miss file update events, and other tech such as docker can trigger heavy CPU trying to make the opposite tradeoff. It also makes IDE/editor integration much easier

alxbarron commented 2 years ago

I was having the same issue, and I was refusing to try this so I was already resigned to run the npm commands through the host's terminal.

I decide to downgrade the npm 8.3.1 version on my local to the default version for node v14.18.1 which is npm 6.14.15, so just for curiosity I tried for the last time after the downgrade to run the installation script through the vvv-vm and this time it worked just fine. So it may worth the shot to validate your node/npm versions betwen your host and the vvv-vm, note that in my case they don't exactly match but the mayor versions do matches.

Before

vvv-vm
- node v14.19.0 w/ npm 6.14.16
host
- node v14.18.1 w/ npm 8.3.1

After

vvv-vm
- node v14.19.0 w/ npm 6.14.16
host
- node v14.18.1 w/ npm 6.14.15

cc: @columbian-chris @tomjn

tomjn commented 2 years ago

ENOENT is a very generic error code, I've encountered this on the host when no VM or VVV is involved because VSCode was running commands. It can even happen due to file ownership or because Explorer was generating a thumbnail preview.

It can happen even if VVV is operating perfectly with no bugs, or if there is no VVV at all. It's a general npm issue, not a VVV issue.

The most effective solution is usually npm -rf node_modules && npm install.

@alxbarron you may find the nvm PR currently in progress interesting, I'd be curious for any feedback you have for it, or just testing

tomjn commented 2 years ago

I'd also suggest minimum versions in package.json:

  "engines": {
    "node": ">=14.17.0",
    "npm": ">=7.18.0"
  },

It's not a good idea to mix major versions of npm, if only for lock file stability in version control

Varying-Vagrant-Vagrants / VVV

npm can't install or update #2529

Before

After