error on ENOENT lstat could use additional context

[othiym23]

When a caller hits this stat failure, it takes considerable sleuthing to figure out why the stat was even happening. Wrapping the error message up with some explanatory text would be very helpful here.

[dylang]

Are you able to repro this with some consistency?

We get it about once every 5 installs on our CI slave, always with the same from from the same package, but we're not sure what's required to have it happen every time. We also have it happen on our local dev boxes, but not as frequently. The error is always from same file.

[isaacs]

@dylang I've literally never seen this. (@othiym23 have you?)

Do you have a package I can try to install where you see this occur? What is the same file that this happens to?

This stat failure, if an ENOENT, would mean that the Writer finished writing its thing, but then after creating the thing, it didn't exist.

If you're writing a symlink, and in follow:true mode (or on an OS without lstat) then it could raise an ENOENT because the target doesn't exist. For anything else, it should be super strange! The only way for that to occur would be if something deleted the file while the fstream.Writer object was in the process of writing to it, which is almost certainly a relevant failure.

[dylang]

I don't have a package you can try, it's our closed-source (for now at least) build system.

The file that seems the problem is angular-bundle/lib/angular-animate.js, which looks similar to this version for bower. The package this file is in happens to get installed 64 times because many dependencies have a dependency or peer dependency on Angular. We theorize reducing the number of copies will reduce the occurrence of this problem.

Here's last part of a log. I don't want to share the entire log because I'd need to make sure it was all safe to share on the interwebs and it's really long.

160695 verbose rebuild bundle /Users/some.developer/workspace/x-web/widget-astro-poc/node_modules/x-web-taskmaster/node_modules/grunt-task-test-harness/node_modules/transform-package/node_modules/x-web-npm/node_modules/npm/node_modules/sha
160696 info build /Users/some.developer/workspace/x-web/widget-astro-poc/node_modules/x-web-taskmaster/node_modules/grunt-task-test-harness/node_modules/transform-package/node_modules/x-web-npm/node_modules/npm/node_modules/sha
160697 info preinstall sha@1.2.4
160698 silly gunzTarPerm extractEntry doc/cli/npm-owner.md
160699 silly gunzTarPerm extractEntry doc/cli/npm-pack.md
160700 verbose linkStuff [ false,
160700 verbose linkStuff   false,
160700 verbose linkStuff   false,
160700 verbose linkStuff   '/Users/some.developer/workspace/x-web/widget-astro-poc/node_modules/x-web-taskmaster/node_modules/grunt-task-test-harness/node_modules/transform-package/node_modules/x-web-npm/node_modules/npm/node_modules' ]
160701 info linkStuff sha@1.2.4
160702 verbose linkBins sha@1.2.4
160703 verbose linkMans sha@1.2.4
160704 verbose rebuildBundles sha@1.2.4
160705 verbose rebuildBundles [ 'readable-stream' ]
160706 info install sha@1.2.4
160707 error Error: ENOENT, lstat '/Users/some.developer/workspace/x-web/widget-astro-poc/node_modules/angular-bundle/lib/angular-animate.js'
160708 error If you need help, you may report this *entire* log,
160708 error including the npm and node versions, at:
160708 error     <http://github.com/npm/npm/issues>
160709 error System Darwin 13.3.0
160710 error command "node" "/Users/some.developer/npm/bin/npm" "install"
160711 error cwd /Users/some.developer/workspace/x-web/widget-astro-poc
160712 error node -v v0.10.24
160713 error npm -v 1.4.16
160714 error path /Users/some.developer/workspace/x-web/widget-astro-poc/node_modules/angular-bundle/lib/angular-animate.js
160715 error fstream_path /Users/some.developer/workspace/x-web/widget-astro-poc/node_modules/angular-bundle/lib/angular-animate.js
160716 error fstream_type File
160717 error fstream_class FileWriter
160718 error code ENOENT
160719 error errno 34
160720 error fstream_stack /Users/some.developer/npm/lib/node_modules/npm/node_modules/fstream/lib/writer.js:284:26
160720 error fstream_stack Object.oncomplete (evalmachine.<anonymous>:107:15)
160721 verbose exit [ 34, true ]

My teammates might have more to share. Thanks for your attention to this, it's been painful having builds frequently fail for seemingly unknown reasons.

[nowells]

@isaacs @othiym23 this manifests itself in the ticket reported here: https://github.com/npm/npm/issues/5841 where I attached the full npm-debug.log.

[othiym23]

@isaacs I haven't seen this happen on my machine since we released 1.4.15 with the fixes around locking. Given the VAST number of issues we see around this, though, I completely believe that there's something fishy going on here.