fs.watch: node[52551] (CarbonCore.framework) FSEventStreamStart: register_with_server: ERROR: f2d_register_rpc() => (null) (-21)

[shama]

I'm getting an error which appears to be directly from libuv while using fs.watch. I'm on OSX 10.8.3 and using node 0.10.5. My ulimit -n is 10480.

I'm able to duplicate the error with the following example (it is a bit contrived but it simulates what is happening with our gruntjs/grunt-contrib-watch module). It only happens when you're watching a larger number of files and another process tries to watch the same files.

I've setup an example in a repo for duplicating the error:

git clone git://github.com/shama/fs-watch-error && cd fs-watch-error
npm install
npm test

---

Here is the code that duplicates the issue:

var fs = require('fs');
var glob = require('glob');
var limit = 9999;
glob('**/*', function(err, files) {
  if (files.length > limit) files = files.slice(0, limit);
  console.log('watching ' + files.length + ' files...');
  files.forEach(function(file) {
    fs.watch(file, function() {});
  });
});

Then run node index.js & node index.js on a large number of files and you should get:

2013-05-13 11:39 node[54911] (CarbonCore.framework) FSEventStreamStart: register_with_server: ERROR: f2d_register_rpc() => (null) (-21)

---

I'm looking into updating our module to avoid this error but I'm reporting it here because I believe it shouldn't error out to the console in that way. Considering that I don't seem to be able to catch that error and handle it more appropriately.

Thanks!

[SimonKaluza]

Scratch that -- installing Node v0.11.14 only cleared up the issue for me temporarily. After a few Node server restarts, the issue began reoccurring on node v 0.11.14. Downgrading to 0.11.13 allowed me to continue working for a while without it returning, but soon enough the error began rearing its head on that installation as well.

[indutny]

@SimonKaluza this is very sad news. I guess FSEventStream is even more broken, than I thought. I'll report it on Apple's bug tracker and see how far we can go from there.

[indutny]

Here is a radar ticket number, just for reference: 18700063 :)

[ziggythehamster]

+1 for me:

keith@Keiths-iMac ~ $ uname -a
Darwin Keiths-iMac.local 13.4.0 Darwin Kernel Version 13.4.0: Sun Aug 17 19:50:11 PDT 2014; root:xnu-2422.115.4~1/RELEASE_X86_64 x86_64

keith@Keiths-iMac ~ $ node --version
v0.10.33

keith@Keiths-iMac ~ $ system_profiler SPSoftwareDataType SPDeveloperToolsDataType
Software:

    System Software Overview:

      System Version: OS X 10.9.5 (13F34)
      Kernel Version: Darwin 13.4.0
      Boot Volume: Macintosh HD
      Boot Mode: Normal
      Computer Name: Keith’s iMac
      User Name: Keith Gable (keith)
      Secure Virtual Memory: Enabled
      Time since boot: 3:54

Developer:

    Developer Information:

      Version: 6.1.1 (6A2008a)
      Location: /Applications/Xcode.app
      Applications:
          Xcode: 6.1.1 (6611)
          Instruments: 6.1 (56160)
      SDKs:
          OS X:
              10.10: (14A382)
              10.9: (13F26)
          iOS:
              8.1: (12B411)
          iOS Simulator:
              8.1: (12B411)

[ziggythehamster]

This Sublime Text 2 configuration snippet seems to have helped:

"folder_exclude_patterns": [".svn", ".git", ".hg", "CVS", "tmp/class-*", "tmp/es_*", "tmp/jshinter*", "tmp/replace_*", "tmp/static_compiler*", "tmp/template_compiler*", "tmp/tree_merger*", "tmp/coffee_script*", "tmp/concat-tmp*", "tmp/export_tree*", "tmp/sass_compiler*"]

[consideRatio]

what @ziggythehamster did with excluding folders in a sublime project helped me as well for my Ember-cli app. When i was running a ember-cli dev-server who as well as sublime also watches files i got this error, but it was solved when i using sublime configurations removed heavy project folders like a tmp folder where massive changes would occur quickly in the file system.

So +1 for using the "folder_exclude_patterns": ["foldername1"] setting in sublime

[kaldrenon]

This does not appear to be fixed in v0.11.14

[Samstiles]

Still occuring for me as well

[jclif]

I'm getting this error quite often. I believe it's being caused by gulp-watch. Also, the error message is sometimes inserted into vim.

[rushkeldon]

We have been struggling with these errors on a project for a while. I don't think it is the number of files that are being watched - I think this error comes from trying to register a watch on the same file more than once. So, make sure your blobs don't overlap in subsequent calls to watch.

Also, if you are knowingly calling watch on the same file(s) it is good to know that gulp.watch returns a watcher that will relay the event every time, so you can chain a '.on' block like so :

GOOD :

 gulp.watch( constants.APP_TEMPLATES, [ 'appTemplates' ] )
    .on( 'change', function( event ){ logChangedFile( event ); }  );

as opposed to the wrong way that was generating about 250 of these errors every time I ran Gulp.

BAD :

gulp.watch( constants.APP_TEMPLATES, logChangedFile );
gulp.watch( constants.APP_TEMPLATES, [ 'appTemplates' ] );

[Sup3rDave]

I still get the same issues in v0.12, pretty frustrating to deal with.

[ziggythehamster]

I'm not 100% sure the problem is Node's fault. I fixed the problem by adding my Broccoli build directory to Sublime Text's and Visual Studio Code's ignore list. I think OS X just has a ridiculously crappy file system watcher.

[nmschulte]

I'm experiencing this issue with recent versions of Node and npm on OS X machinery.

On OS X 10.10.4 (14E46):

$ node --version
v0.10.35
$ npm --version
2.13.5

On OS X 10.10.5:

$ node -v
v0.12.7
$ npm -v
2.14.3

On my machine running GNU/Linux (Debian Sid; Linux 4.x), these issues don't exist:

$ node --version
v0.12.4
$ npm --version
2.14.1

This makes cross-environment/platform development a real pain: webpack's watching implementation doesn't always pick up on changes in Linux (possibly more), and the work-around is to use the "old" watching plugin, which suffers from these issues. It's an un-winnable game!

[Sup3rDave]

I ultimately found that my issue was Sublime contributing to watching too many files on OSX's file system. If I shut down Sublime I was immediately able to watch with no issue, then I could start Sublime up again and continue. Not the greatest solution admittedly, but it works at least.

[nmschulte]

@Sup3rDave, so, is this simply an incompatibility issue with OS X? If so, it's still an issue that ought to be dealt with, even if that simply means training OS X users how their OS works... (e.g. ulimit or w/e resource management issue is getting in the way).

Were you able to track what about OS X is causing this?

[Sup3rDave]

@nmschulte no, I was never able to figure out specifically what it was about OSX other than I guess some issue with its FSEvent stuff which apparently Gulp/Sublime(and probably other things) lean on for watching. I was mid project when I found the workaround, so I was more concerned with getting past it anyway :)

[nmschulte]

@Sup3rDave, thanks for the followup.

Well, it seems that this issue will not ever see the light of day, given the recent change of repositories to nodejs/node, and that this issue is closed here in the archive repository. I'm going to confirm the issue still exists with the latest versions of things on OS X, and if so create a new issue on the new repository with the new information and referencing this old one.

[ziggythehamster]

I think the problem is that there's a finite number of FSEvent watchers you can have among all apps using CarbonCore, and Sublime and others aren't very polite about not using all of the watchers. I added my Ember CLI temp and build directories to Sublime's ignore list and it solved my issue. I think the problem would be resolved by having Node use Cocoa's file system watcher, but that sounds like a huge pain in the ass.

[nmschulte]

Carbon Core is deprecated; sounds like Node.js should switch to use Cocoa, regardless of how much of a pain it is. I have to imagine this effort is already underway, considering I don't imagine Node.js plans to drop support for OS X...

https://developer.apple.com/library/mac/releasenotes/General/CarbonCoreDeprecations/index.html#//apple_ref/doc/uid/TP40012224

That said, I'm not entirely certain node is using the deprecated Carbon Core API, or at least perhaps not with my specific incantation of this issue:

https://developer.apple.com/library/mac/documentation/Darwin/Conceptual/FSEvents_ProgGuide/Introduction/Introduction.html#//apple_ref/doc/uid/TP40005289

(FSEvents.framework) FSEventStreamStart: register_with_server: ERROR: f2d_register_rpc() => (null) (-21)

[ziggythehamster]

I think they're using FSEvents, and FSEvents internally uses CarbonCore. Apple: Do as I say, not as I do, right? :)

I mean, iTunes was/is Carbon64 (something they never released for anyone else) after all.