Bug: Maximum call stack size exceeded

[dvargas92495]

I added new DuplicatesPlugin() to my webpack config (node 12.8.1, npm 6.10.2, webpack 4.39.1) and after running webpack I eventually got RangeError: Maximum call stack size exceeded.

Here's the stack trace (I trimmed the path to project out)

    at String.split (<anonymous>)
    at Object.exports.toPosixPath (node_modules/inspectpack/lib/util/files.js:34:38)
    at node_modules/inspectpack/lib/util/dependencies.js:44:54
    at Array.filter (<anonymous>)
    at _recurseDependencies (node_modules/inspectpack/lib/util/dependencies.js:143:10)
    at node_modules/inspectpack/lib/util/dependencies.js:187:32
    at Array.map (<anonymous>)
    at _recurseDependencies (node_modules/inspectpack/lib/util/dependencies.js:178:10)
    at node_modules/inspectpack/lib/util/dependencies.js:187:32
    at Array.map (<anonymous>)
    at _recurseDependencies (node_modules/inspectpack/lib/util/dependencies.js:178:10)
    at node_modules/inspectpack/lib/util/dependencies.js:187:32
    at Array.map (<anonymous>)
    at _recurseDependencies (node_modules/inspectpack/lib/util/dependencies.js:178:10)
    at node_modules/inspectpack/lib/util/dependencies.js:187:32
    at Array.map (<anonymous>)
    at _recurseDependencies (node_modules/inspectpack/lib/util/dependencies.js:178:10)
    ...

Running with verbose: true didn't give any extra insight

I know probably informing you what's in my package.json would help debug what's going on, but we have a lot of packages so it's tough to know what combination of packages is causing this issue. Is there a way for me to debug this further so that I know which combination of packages is responsible?

[ryan-roemer]

Thanks for the report!

Can you try adding these console logs to your installed node_modules/inspectpack file?

diff --git a/lib/util/dependencies.js b/lib/util/dependencies.js
index 139e847..e3c4bc4 100644
--- a/lib/util/dependencies.js
+++ b/lib/util/dependencies.js
@@ -184,6 +184,7 @@ const _recurseDependencies = ({ filePath, foundMap, names, pkgMap, pkgsFilter, }
         // hits to have this mutation step avoided since we manually return
         // `[]` on a cache hit.
         if (pkgNames.length) {
+            console.log("TODO RECURSE DEPS", { filePath: pkg.filePath });
             pkg.dependencies = _recurseDependencies({
                 filePath: pkg.filePath,
                 foundMap: _foundMap,
@@ -199,6 +200,11 @@ const _identifyCircularRefs = (pkg, refPath) => {
     const _refPath = refPath || [];
     // Detect circular and short-circuit.
     const circRef = _refPath.find((ref) => pkg === ref);
+    console.log("TODO CIRCULAR DEPS", {
+        circRef: (circRef || {}).filePath,
+        pkg: pkg.filePath,
+        _refPath: _refPath.map((o) => o.name)
+    });
     if (circRef) {
         return {
             isCircular: true,

You're going to get a lot of output, but I'm hoping the recursion issue gives some more information about what dependency combination is triggering this. Then, if you could:

1. Report back with the log information (and go ahead and sanitize if sensitive) of a recursion pattern if there is one before the stack overflow.
2. If you can give me a file structure layout of where the implicated packages are placed relative to project root.
3. Check and let me know if any of the packages are symlinks or something that's not an ordinary file.

If this doesn't work / produce some insight, our next step is probably producing a huge stats object that I can work with (which will mean you'd have to manually sanitize out all your private code, since it contains the sources -- I can give you guidance, and we have had folks do this before when the needed thing to debug an issue, but it is usually a bit of a pain...)

[dvargas92495]

ok so update

here's the node modules that are recursing:

TODO RECURSE DEPS {
  filePath: '.../node_modules/d'
}
TODO RECURSE DEPS {
  filePath: '.../node_modules/es5-ext'
}
TODO RECURSE DEPS {
  filePath: '.../node_modules/es6-symbol'
}
TODO RECURSE DEPS {
  filePath: '.../node_modules/d'
}
TODO RECURSE DEPS {
  filePath: '.../node_modules/es5-ext'
}
TODO RECURSE DEPS {
  filePath: '.../node_modules/es6-symbol'
}

and these three repeat forever.

Looking at various package.json's, the only package that seems to reference these three are this memoizee package: https://github.com/medikoo/memoizee/blob/master/package.json. It also seems like it's an actual circular dependency: es5-ext depends on es6-symbol which depends on d which depends on es5-ext which makes sense. The same github user has developed all 4, so maybe I should create an issue under the memoizee repo about the circular dependency?

to answer 2), the package is only used in one file lib/memoizee/index (looks like we wrap the library)

[ryan-roemer]

Thanks for the research work!

inspectpack should handle circular dependencies (which usually result from npm/yarn flattening). Is there any chance you can identify a small code snippet that triggers those?

[dvargas92495]

when you say identify a small code snippet do you mean in dependencies.js or the snippet in our source that uses the memoizee library?

[ryan-roemer]

Ah, good question!

Your package.json:dependencies|peerDependencies|devDependencies for anything that relates to those (transitive or direct dependencies).

And in your source code, any direct imports or requires.

[dvargas92495]

ah got it! package.json:

dependencies: {
    ...
    "memoizee": "0.4.14",
    ...
}

in app/lib/memoizee/index.js:

import memoize from 'memoizee';

export default (fn, config) => {
    const enhancedConfig = {
        // ...someExtraStuff,
        ...config
    }
    return memoize(fn, enhancedConfig);
};

[ryan-roemer]

Awesome -- thanks!

I've been able to confirm the bug with a regression test at: https://github.com/FormidableLabs/inspectpack/compare/bug/recurse-deps (WIP branch)

Reproduction:

$ WEBPACK_VERSION=4 \
  WEBPACK_MODE=development \
  WEBPACK_CWD=../../test/fixtures/circular-deps \
  yarn build-test-wp

# ... <SNIPPED> ...
TODO RECURSE DEPS { filePath: '/Users/rye/scm/fmd/inspectpack/node_modules/es5-ext' }
TODO RECURSE DEPS { filePath: '/Users/rye/scm/fmd/inspectpack/node_modules/es6-symbol' }
TODO RECURSE DEPS { filePath: '/Users/rye/scm/fmd/inspectpack/node_modules/d' }
TODO RECURSE DEPS { filePath: '/Users/rye/scm/fmd/inspectpack/node_modules/es5-ext' }
TODO RECURSE DEPS { filePath: '/Users/rye/scm/fmd/inspectpack/node_modules/es6-symbol' }
TODO RECURSE DEPS { filePath: '/Users/rye/scm/fmd/inspectpack/node_modules/d' }
TODO RECURSE DEPS { filePath: '/Users/rye/scm/fmd/inspectpack/node_modules/es5-ext' }
TODO RECURSE DEPS { filePath: '/Users/rye/scm/fmd/inspectpack/node_modules/es6-symbol' }
TODO RECURSE DEPS { filePath: '/Users/rye/scm/fmd/inspectpack/node_modules/d' }
RangeError: Maximum call stack size exceeded
    at formatRaw (internal/util/inspect.js:814:15)
    at formatValue (internal/util/inspect.js:591:10)
    at inspect (internal/util/inspect.js:221:10)
    at formatWithOptions (internal/util/inspect.js:1693:40)
    at Object.Console.<computed> (internal/console/constructor.js:272:10)
    at Object.log (internal/console/constructor.js:282:61)
    at /Users/rye/scm/fmd/inspectpack/lib/util/dependencies.js:187:21
    at Array.map (<anonymous>)
    at _recurseDependencies (/Users/rye/scm/fmd/inspectpack/lib/util/dependencies.js:178:10)
    at /Users/rye/scm/fmd/inspectpack/lib/util/dependencies.js:188:32
    at Array.map (<anonymous>)
    at _recurseDependencies (/Users/rye/scm/fmd/inspectpack/lib/util/dependencies.js:178:10)
    at /Users/rye/scm/fmd/inspectpack/lib/util/dependencies.js:188:32
    at Array.map (<anonymous>)
    at _recurseDependencies (/Users/rye/scm/fmd/inspectpack/lib/util/dependencies.js:178:10)
    at /Users/rye/scm/fmd/inspectpack/lib/util/dependencies.js:188:32
    at Array.map (<anonymous>)
    at _recurseDependencies (/Users/rye/scm/fmd/inspectpack/lib/util/dependencies.js:178:10)
    at /Users/rye/scm/fmd/inspectpack/lib/util/dependencies.js:188:32
    at Array.map (<anonymous>)
    at _recurseDependencies (/Users/rye/scm/fmd/inspectpack/lib/util/dependencies.js:178:10)
    at /Users/rye/scm/fmd/inspectpack/lib/util/dependencies.js:188:32
    at Array.map (<anonymous>)
    at _recurseDependencies (/Users/rye/scm/fmd/inspectpack/lib/util/dependencies.js:178:10)
    at /Users/rye/scm/fmd/inspectpack/lib/util/dependencies.js:188:32
    at Array.map (<anonymous>)
    at _recurseDependencies (/Users/rye/scm/fmd/inspectpack/lib/util/dependencies.js:178:10)
    at /Users/rye/scm/fmd/inspectpack/lib/util/dependencies.js:188:32
    at Array.map (<anonymous>)
    at _recurseDependencies (/Users/rye/scm/fmd/inspectpack/lib/util/dependencies.js:178:10)