broccoli serve with mergeTrees runs out of memory

[jamesarosen]

When using a mergeTrees (v1.1.1) of funnels (v1.0.0 or v1.0.1), broccoli serve runs out of memory and crashes.

Simplest Brocfile.js:

var funnel = require('broccoli-funnel');
var mergeTrees = require('broccoli-merge-trees');

module.exports = mergeTrees([
  funnel('.', { include: [ 'tests/**/*.js' ] })
]);

When I run broccoli build dist, everything works fine. When I run broccoli serve, I get

Serving on http://localhost:4200

Warning: failed to stat tmp/funnel-input_base_path-DF6uadv1.tmp/0/tmp/funnel-input_base_path-DF6uadv1.tmp/0/tmp/funnel-input_base_path-DF6uadv1.tmp/0/tmp/funnel-input_base_path-DF6uadv1.tmp/0/tmp/funnel-input_base_path-DF6uadv1.tmp/0/tmp/funnel-input_base_path-DF6uadv1.tmp/0/tmp/funnel-input_base_path-DF6uadv1.tmp/0/tmp/funnel-input_base_path-DF6uadv1.tmp/0/tmp/funnel-input_base_path-DF6uadv1.tmp/0/tmp/funnel-input_base_path-DF6uadv1.tmp/0/tmp/funnel-input_base_path-DF6uadv1.tmp/0/tmp/funnel-input_base_path-DF6uadv1.tmp/0/tmp/funnel-input_base_path-DF6uadv1.tmp/0/tmp/funnel-input_base_path-DF6uadv1.tmp/0/tmp/funnel-input_base_path-DF6uadv1.tmp/0/tmp/funnel-input_base_path-DF6uadv1.tmp/0/tmp/funnel-input_base_path-DF6uadv1.tmp/0/tmp/funnel-input_base_path-DF6uadv1.tmp/0/tmp/funnel-input_base_path-DF6uadv1.tmp/0/node_modules/broccoli-babel-transpiler/node_modules/babel-core/node_modules/regenerator/node_modules/defs/node_modules/yargs/node_modules/cliui/node_modules/center-align/node_modules/align-text/node_modules/kind-of/node_modules

<--- Last few GCs --->

   20320 ms: Scavenge 1402.9 (1457.7) -> 1402.9 (1457.7) MB, 9.3 / 0 ms (+ 34.5 ms in 1 steps since last GC) [allocation failure] [incremental marking delaying mark-sweep].
   21740 ms: Mark-sweep 1402.9 (1457.7) -> 1401.6 (1457.7) MB, 1420.3 / 0 ms (+ 295.9 ms in 1199 steps since start of marking, biggest step 34.5 ms) [last resort gc].
   23137 ms: Mark-sweep 1401.6 (1457.7) -> 1401.5 (1457.7) MB, 1397.5 / 0 ms [last resort gc].

<--- JS stacktrace --->

==== JS stack trace =========================================

Security context: 0x43f935e3ac1 <JS Object>
    2: keysForTree(aka keysForTree) [/Users/jamesarosen/Desktop/graphs/node_modules/broccoli/node_modules/broccoli-kitchen-sink-helpers/index.js:49] [pc=0x1d274e92b223] (this=0x43f93504189 <undefined>,fullPath=0x6d8b44e2d19 <String[983]: tmp/funnel-input_base_path-DF6uadv1.tmp/0/tmp/funnel-input_base_path-DF6uadv1.tmp/0/tmp/funnel-input_base_path-DF6uadv1.tmp/0/tmp/funnel-input_base_path-DF6uadv...

FATAL ERROR: CALL_AND_RETRY_LAST Allocation failed - process out of memory
Abort trap: 6

The warning seems to have some sort of infinite loop in the path, which suggests a reason for the out-of-memory error.

I've tried this on node v0.12.0 and v5.7.1.

[rwjblue]

You can't watch the directory that contains 'tmp'. If you do, it will trigger recursive rebuilds as each file in tmp is also triggering a rebuild.

Change '.' to use 'tests' and update your includes to accommodate.

[jamesarosen]

I assumed the include (which does not include tmp/) would have prevented that.

I'll try funnel('tests', { destDir: 'tests' })

[rwjblue]

Include is only restricting what is used, not what is watched.

[jamesarosen]

Success. Thanks :)