"Maximum call stack size exceeded" in streamChunks*

[WolfspiritM]

Using storybook with webpack5 we recently ended up getting "Maximum call stack size exceeded" when we try to build the storybook project.

    RangeError: Maximum call stack size exceeded
    at String.match (<anonymous>)
    at streamChunksOfRawSource (C:\src\node_modules\webpack\node_modules\webpack-sources\lib\helpers\streamChunksOfRawSource.js:14:25)
    at module.exports (C:\src\node_modules\webpack\node_modules\webpack-sources\lib\helpers\streamChunksOfRawSource.js:40:5)
    at RawSource.streamChunks (C:\src\node_modules\webpack\node_modules\webpack-sources\lib\RawSource.js:56:10)
    at module.exports (C:\src\node_modules\webpack\node_modules\webpack-sources\lib\helpers\streamChunks.js:13:17)
    at streamAndGetSourceAndMap (C:\src\node_modules\webpack\node_modules\webpack-sources\lib\helpers\streamAndGetSourceAndMap.js:27:53)
    at CachedSource.streamChunks (C:\src\node_modules\webpack\node_modules\webpack-sources\lib\CachedSource.js:208:35)
    at module.exports (C:\src\node_modules\webpack\node_modules\webpack-sources\lib\helpers\streamChunks.js:13:17)
    at ReplaceSource.streamChunks (C:\src\node_modules\webpack\node_modules\webpack-sources\lib\ReplaceSource.js:176:44)
    at module.exports (C:\src\node_modules\webpack\node_modules\webpack-sources\lib\helpers\streamChunks.js:13:17)
    at ConcatSource.streamChunks (C:\src\node_modules\webpack\node_modules\webpack-sources\lib\ConcatSource.js:112:55)
    at module.exports (C:\src\node_modules\webpack\node_modules\webpack-sources\lib\helpers\streamChunks.js:13:17)
    at streamAndGetSourceAndMap (C:\src\node_modules\webpack\node_modules\webpack-sources\lib\helpers\streamAndGetSourceAndMap.js:27:53)
    at CachedSource.streamChunks (C:\src\node_modules\webpack\node_modules\webpack-sources\lib\CachedSource.js:208:35)
    at module.exports (C:\src\node_modules\webpack\node_modules\webpack-sources\lib\helpers\streamChunks.js:13:17)
    at ConcatSource.streamChunks (C:\src\node_modules\webpack\node_modules\webpack-sources\lib\ConcatSource.js:112:55)

It seems to be caused by the regex used for the match. I have seen it in two different spots. The other one was:

UnhandledPromiseRejectionWarning: RangeError: Maximum call stack size exceeded
    at String.match (<anonymous>)
    at streamChunksOfSourceMapLinesFull (\node_modules\webpack\node_modules\webpack-sources\lib\helpers\streamChunksOfSourceMap.js:167:23)
    at module.exports (\node_modules\webpack\node_modules\webpack-sources\lib\helpers\streamChunksOfSourceMap.js:405:6)
    at nameIndexMapping.<computed> (\node_modules\webpack\node_modules\webpack-sources\lib\helpers\streamChunksOfCombinedSourceMap.js:271:5)
    at streamChunksOfSourceMapLinesFinal (\node_modules\webpack\node_modules\webpack-sources\lib\helpers\streamChunksOfSourceMap.js:331:3)
    at module.exports (\node_modules\webpack\node_modules\webpack-sources\lib\helpers\streamChunksOfSourceMap.js:398:6)
    at streamChunksOfCombinedSourceMap (\node_modules\webpack\node_modules\webpack-sources\lib\helpers\streamChunksOfCombinedSourceMap.js:54:9)
    at SourceMapSource.streamChunks (\node_modules\webpack\node_modules\webpack-sources\lib\SourceMapSource.js:197:11)
    at exports.getSourceAndMap (\node_modules\webpack\node_modules\webpack-sources\lib\helpers\getFromStreamChunks.js:17:33)
    at SourceMapSource.sourceAndMap (\node_modules\webpack\node_modules\webpack-sources\lib\SourceMapSource.js:188:10)
    at getTaskForFile (\node_modules\webpack\lib\SourceMapDevToolPlugin.js:84:30)
    at \node_modules\webpack\lib\SourceMapDevToolPlugin.js:272:22
    at \node_modules\webpack\lib\Cache.js:93:5
    at Hook.eval [as callAsync] (eval at create (\node_modules\webpack\node_modules\tapable\lib\HookCodeFactory.js:33:10), <anonymous>:6:1)
    at Cache.get (\node_modules\webpack\lib\Cache.js:75:18)
    at ItemCacheFacade.get (\node_modules\webpack\lib\CacheFacade.js:117:15)

At first I thought it might be cause of a big chunk and source file but I tried to output "source" prior to the "source.match" and it seems to only contain:

/* module decorator */ module = __webpack_require__.hmd(module);\n

I'm not sure how to create a reproducable way for that issues however it vanishes when downgrading to webpack ~5.46.0 and whenever I upgrade back to ~5.47.0 the error reappears.

[ihor-panasiuk95]

@alexander-akait

[alexander-akait]

You have mixed webpack versions...

[dsm0880]

@alexander-akait @Bjeaurn I'm receiving the same issue. Below is my stack trace and npm ls webpack. I'm running Angular 12.2.5.

`node --trace-warnings ./node_modules/@angular/cli/bin/ng serve

⠧ Generating browser application bundles (phase: sealing)...(node:14108) UnhandledPromiseRejectionWarning: RangeError: Maximum call stack size exceeded at String.match () at checkOriginalContent (\node_modules\webpack\node_modules\webpack-sources\lib\ReplaceSource.js:166:23) at \node_modules\webpack\node_modules\webpack-sources\lib\ReplaceSource.js:358:9 at onMapping (\node_modules\webpack\node_modules\webpack-sources\lib\helpers\streamChunksOfSourceMap.js:78:5) at readMappings (\node_modules\webpack\node_modules\webpack-sources\lib\helpers\readMappings.js:61:6) at streamChunksOfSourceMapFull (\node_modules\webpack\node_modules\webpack-sources\lib\helpers\streamChunksOfSourceMap.js:152:2) at module.exports (\node_modules\webpack\node_modules\webpack-sources\lib\helpers\streamChunksOfSourceMap.js:389:6) at SourceMapSource.streamChunks (\node_modules\webpack\node_modules\webpack-sources\lib\SourceMapSource.js:211:11) at module.exports (\node_modules\webpack\node_modules\webpack-sources\lib\helpers\streamChunks.js:13:17) at ReplaceSource.streamChunks (\node_modules\webpack\node_modules\webpack-sources\lib\ReplaceSource.js:176:44) at module.exports (\node_modules\webpack\node_modules\webpack-sources\lib\helpers\streamChunks.js:13:17) at streamAndGetSourceAndMap (\node_modules\webpack\node_modules\webpack-sources\lib\helpers\streamAndGetSourceAndMap.js:27:53) at CachedSource.streamChunks (\node_modules\webpack\node_modules\webpack-sources\lib\CachedSource.js:208:35) at module.exports (\node_modules\webpack\node_modules\webpack-sources\lib\helpers\streamChunks.js:13:17) at ConcatSource.streamChunks (\node_modules\webpack\node_modules\webpack-sources\lib\ConcatSource.js:112:55) at module.exports (\node_modules\webpack\node_modules\webpack-sources\lib\helpers\streamChunks.js:13:17) at emitUnhandledRejectionWarning (internal/process/promises.js:168:15) at processPromiseRejections (internal/process/promises.js:247:11) at processTicksAndRejections (internal/process/task_queues.js:96:32) (node:14108) RangeError: Maximum call stack size exceeded at String.match () at checkOriginalContent (\node_modules\webpack\node_modules\webpack-sources\lib\ReplaceSource.js:166:23) at \node_modules\webpack\node_modules\webpack-sources\lib\ReplaceSource.js:358:9 at onMapping (\node_modules\webpack\node_modules\webpack-sources\lib\helpers\streamChunksOfSourceMap.js:78:5) at readMappings (\node_modules\webpack\node_modules\webpack-sources\lib\helpers\readMappings.js:61:6) at streamChunksOfSourceMapFull (\node_modules\webpack\node_modules\webpack-sources\lib\helpers\streamChunksOfSourceMap.js:152:2) at module.exports (\node_modules\webpack\node_modules\webpack-sources\lib\helpers\streamChunksOfSourceMap.js:389:6) at SourceMapSource.streamChunks (\node_modules\webpack\node_modules\webpack-sources\lib\SourceMapSource.js:211:11) at module.exports (\node_modules\webpack\node_modules\webpack-sources\lib\helpers\streamChunks.js:13:17) at ReplaceSource.streamChunks (\node_modules\webpack\node_modules\webpack-sources\lib\ReplaceSource.js:176:44) at module.exports (\node_modules\webpack\node_modules\webpack-sources\lib\helpers\streamChunks.js:13:17) at streamAndGetSourceAndMap (\node_modules\webpack\node_modules\webpack-sources\lib\helpers\streamAndGetSourceAndMap.js:27:53) at CachedSource.streamChunks (\node_modules\webpack\node_modules\webpack-sources\lib\CachedSource.js:208:35) at module.exports (\node_modules\webpack\node_modules\webpack-sources\lib\helpers\streamChunks.js:13:17) at ConcatSource.streamChunks (\node_modules\webpack\node_modules\webpack-sources\lib\ConcatSource.js:112:55) at module.exports (\node_modules\webpack\node_modules\webpack-sources\lib\helpers\streamChunks.js:13:17) (node:14108) [DEP0018] DeprecationWarning: Unhandled promise rejections are deprecated. In the future, promise rejections that are not handled will terminate the Node.js process with a non-zero exit code. at emitDeprecationWarning (internal/process/promises.js:180:11) at processPromiseRejections (internal/process/promises.js:249:13) at processTicksAndRejections (internal/process/task_queues.js:96:32)`

npm ls webpack on project

`>npm ls webpack

`-- @angular-devkit/build-angular@12.2.5 `-- webpack@5.50.0`

[alan-agius4]

I recommand that a minimal reproduction is provided, as otherwise it would be hard to determine the root cause of the problem.

While I cannot talk on behalf @alexander-akait, I doubt that this can be addressed without a reproduction and a good understanding to what is causing this issue.

[alexander-akait]

Yes, will be great if somebody provide variables (using debug utils) from https://github.com/webpack/webpack-sources/blob/main/lib/ReplaceSource.js#L145 before error was trowed (maybe log them in file) and provide them here

[Bjeaurn]

I could probably get that sorted, stand by!

[alexander-akait]

Also you can invite me in project (if need I can sing NDA) and will investigate this, so you don't need publish project for public (it makes no sense for me to steal a project or code :smile: )

[Bjeaurn]

I messaged you privately about that. In the meanwhile, I think I found a thing!

So I added a bunch of logs to give me an idea what's going on and where it's breaking. I made screenshots for your enjoyment.

I ran into a very long file that had it's sourceMapping inlined, which is where it seemed to break at. This file is an external plugin called survey-angular which seems to consume SurveyJS. Weirdly enough, I can find no usage of the plugin in our actual application, it's also not in the package.json as a listed dependency. But when I do npm uninstall survey-angular the app won't compile/build anymore. I'll dive into this separately.

So this is the start of that, you can see some of my logging around it.

You can see the start of the inlined sourceMap for this plugin.

And here's the end of it where it breaks.

I went into the plugin and removed the inlined sourceMap; now the build succeeds!

What this means exactly for webpack-sources I find a little hard to judge. It might be adding a detection for inlined sourceMaps and shortcircuiting the streamChunks? @alexander-akait

[alexander-akait]

Good catch, we need to look at this deeply, I think we found a place with the problem, pinned, will finish one job and return to this (today, in near future)

[Bjeaurn]

I'm ecstatic. If you need any additional information or logging, or want me to test a new version of the plugin: Feel free to ask!

[alexander-akait]

@Bjeaurn What is Node.js version?

[Bjeaurn]

v12.20.1

Running the same procedure now with v14.17.0. Looks like on 14.17 the package resolving works a bit differently, I'm struggling to get the right 3.2.0 version of webpack-sources to be used because of the complicated dependencies as we saw earlier in this thread.

I'll take a deeper look if required :).

[alexander-akait]

I have good news, reproduced locally, but here same problem:

TypeError: source.match is not a function

      at streamChunksOfRawSource (node_modules/webpack-sources/lib/helpers/streamChunksOfRawSource.js:15:25)
      at Object.<anonymous>.module.exports (node_modules/webpack-sources/lib/helpers/streamChunksOfRawSource.js:40:9)
      at RawSource.streamChunks (node_modules/webpack-sources/lib/RawSource.js:56:10)
      at Object.<anonymous>.module.exports (node_modules/webpack-sources/lib/helpers/streamChunks.js:13:17)
      at ConcatSource.streamChunks (node_modules/webpack-sources/lib/ConcatSource.js:112:55)
      at Object.<anonymous>.exports.getSourceAndMap (node_modules/webpack-sources/lib/helpers/getFromStreamChunks.js:17:33)
      at ConcatSource.sourceAndMap (node_modules/webpack-sources/lib/ConcatSource.js:94:10)
      at getTaskForFile (node_modules/webpack/lib/SourceMapDevToolPlugin.js:84:30)
      at node_modules/webpack/lib/SourceMapDevToolPlugin.js:272:22
      at node_modules/webpack/lib/Cache.js:93:5

Will try to do minimum reproducible test repo in near future, also found another bug :smile: Spent ~2 hours :disappointed:

[Bjeaurn]

You absolute legend! If I can help you let me know right? Happy to test out things. Looking forward to the reproduction.

[alexander-akait]

I am reproduce the problem modification source code, but I can't reproduce it without modification, we have checks to protect this behavior...

[Bjeaurn]

The problem is with the maximum call stack size, is there a check against that?

[alexander-akait]

hm, let's try another thing - https://github.com/webpack/webpack-sources/blob/main/lib/helpers/streamChunksOfRawSource.js#L14, can you provide source value before throwing (and upload file here)?

[Bjeaurn]

On it! Which version do you want me to try this with? Node 14 and Webpack-sources 3.2.0 or 1.4.*?

[alexander-akait]

Any Node.js version and 3.2.0

[Bjeaurn]

It is the same as in the previous screenshots; just the file, no additional values. The inline sourceMap is in there and breaks the call stack maximum.

[alexander-akait]

Yes, but I need content of this file, I think something wrong inside, we have /(?:[^\n]+\n?|\n)/g regexp, and something bad with newlines in file I think, small example the same problem without webpack:

var patt = /(?:\/\/|\/\*)[@#][ \t]+sourceMappingURL=data:(?:application|text)\/json;base64,((?:[A-Za-z0-9+\/]{4})*(?:[A-Za-z0-9+\/]{2}==|[A-Za-z0-9+\/]{3}=)?)(?:[ \t]*\*\/)?$/mg;
var source = '//# sourceMappingURL=data:application/json;base64,'
var len = 4473955
while (source.length < len){
    source += 'ICAgICAgICApXG4gICAgICAgIClcbiAgICAgIClcbi'
}
source = source.substring(0,len)
patt.test(source)

As you can see here example with sourceMappingURL too :smile:

[Bjeaurn]

survey-angular.txt

It's basically the whole file that you'd get if you npm i survey-angular and look at the main .js file in there for reference. This is the exact copied version from the logging tho.

The interesting part from that line would be:

}); //# sourceMappingURL=data:application/json;charset=utf-8;base64,eyJ2ZXJzaW9uIjozLCJzb3VyY2VzIjpbIndlYnBhY2s6Ly9TdXJ2ZXkvd2VicGFjay91bml2ZXJzYWxNb2R1bGVEZWZpbml0aW9uIiwid2VicGFjazovL1N1cnZleS93ZWJwYWNrL

[alexander-akait]

Weird, very, tried different cases but no luck to reproduce :disappointed:

[Bjeaurn]

I think the problem is a combination of a big file like this, in combination with a very large repository.

The streamChunks will probably keep adding to the Maximum call stack size problem as it is recursive. Maybe if you run the same file 10 times in a row in the same build or something? Just an idea.

[alexander-akait]

Tried, no luck...

[alexander-akait]

@Bjeaurn Are you sure provided file is the same as here https://github.com/webpack/webpack-sources/blob/main/lib/helpers/streamChunksOfRawSource.js#L14 before you got error (i.e. throw), because I can't reproduce, tried very very very different things , just modify:

console.log("----");
console.log(source);
console.log("----");
const matches = source.match(MATCH_LINES_REGEX);

and run your build using npm run build &> logs (npm run build can be another in your case, just run compilation when the problem happens)

[Bjeaurn]

That's exactly what I did haha.

I'm happy to run it again if you need, but I'm afraid that's all I got...

Edit: running it and uploading the whole logs file.

[alexander-akait]

@Bjeaurn Can you try to use --stack-trace-limit 20 or even more (you can do using NODE_OPTIONS='--stack-trace-limit 20')? I still think something wrong in env...

[Bjeaurn]

Sure thing. Running with that, I got a private logs file available for you. Let me do that as well.

[Bjeaurn]

@alexander-akait and myself have spend a few hours debugging over the weekend. I recall that he had a local reproduction now but also spotted an additional issue that may be related to this.

I'm not exactly sure on the status and if he has enough to think about a fix.

I'll stay available if additional testing or logging is needed or new versions need to be tested. You know where to find me!

[alexander-akait]

@Bjeaurn Yep, we still investigate this, I am busy on today/tomorrow and then return to this again

[sokra]

Could you try if https://github.com/webpack/webpack-sources/releases/tag/v3.2.1 fixes the problem. It avoids using RegExp for splitting into lines.

[Bjeaurn]

I'll give it a try, although it might be a tad tricky to get the dependencies right as they're "implicit" through other projects. But I'll find a way. Will report back.

Edit: First pass seems to work running webpack-sources@3.2.1 as a direct dependency.

[Bjeaurn]

Update: 2nd pass with caching completely removed also passed. Look like this is a solid fix of the weird edge cases we were running into.

[alexander-akait]

Great! I think we can close it, if somebody faced with this again, please open an issue, hope we fixed all edge cases, feel free to feedback

[dsm0880]

@alexander-akait Currently webpack has a dependency on webpack-sources as follows: "webpack-sources": "^3.2.0"

Does this mean removing my package.lock.json and re-installing webpack should install the latest webpack-sources given you have published a new release of webpack-sources? Just want to make sure I understand and can test appropriately.

[alexander-akait]

Yes, anyway you have npm update (should update all deps)

[dsm0880]

@alexander-akait @Bjeaurn It worked! Thank you both!

[lastnigtic]

Could you try if https://github.com/webpack/webpack-sources/releases/tag/v3.2.1 fixes the problem. It avoids using RegExp for splitting into lines.

May I ask in what situations there will be problems when using RegExp for splitting into lines？

[gardengeek99]

I am still getting this using webpack-sources 3.2.1. Only solution is to backup.

[alexander-akait]

It means you still on old verison, run npm ls webpack-sources

[gardengeek99]

npm ls webpack-sources

apollo-studio@0.0.1 C:\apollo-studio\apollo-studio +-- webpack-sources@3.2.1 -- webpack@5.64.0 -- webpack-sources@3.2.1 deduped

[alexander-akait]

We need reproducible test repo...

[gardengeek99]

That might take awhile. Stack trace might be helpful though.

[webpack-cli] HookWebpackError: Maximum call stack size exceeded
    at makeWebpackError (C:\apollo-studio\apollo-studio\node_modules\webpack\lib\HookWebpackError.js:48:9)
    at C:\apollo-studio\apollo-studio\node_modules\webpack\lib\Compilation.js:3055:12
    at eval (eval at create (C:\apollo-studio\apollo-studio\node_modules\tapable\lib\HookCodeFactory.js:33:10), <anonymous>:68:1)
-- inner error --
RangeError: Maximum call stack size exceeded
    at String.match (<anonymous>)
    at OriginalSource.streamChunks (C:\apollo-studio\apollo-studio\node_modules\webpack-sources\lib\OriginalSource.js:64:32)
    at module.exports (C:\apollo-studio\apollo-studio\node_modules\webpack-sources\lib\helpers\streamChunks.js:13:17)
    at ReplaceSource.streamChunks (C:\apollo-studio\apollo-studio\node_modules\webpack-sources\lib\ReplaceSource.js:175:44)
    at module.exports (C:\apollo-studio\apollo-studio\node_modules\webpack-sources\lib\helpers\streamChunks.js:13:17)
    at ConcatSource.streamChunks (C:\apollo-studio\apollo-studio\node_modules\webpack-sources\lib\ConcatSource.js:112:55)
    at module.exports (C:\apollo-studio\apollo-studio\node_modules\webpack-sources\lib\helpers\streamChunks.js:13:17)
    at streamAndGetSourceAndMap (C:\apollo-studio\apollo-studio\node_modules\webpack-sources\lib\helpers\streamAndGetSourceAndMap.js:27:53)
    at CachedSource.streamChunks (C:\apollo-studio\apollo-studio\node_modules\webpack-sources\lib\CachedSource.js:208:35)
    at module.exports (C:\apollo-studio\apollo-studio\node_modules\webpack-sources\lib\helpers\streamChunks.js:13:17)
caused by plugins in Compilation.hooks.processAssets
RangeError: Maximum call stack size exceeded
    at String.match (<anonymous>)
    at OriginalSource.streamChunks (C:\apollo-studio\apollo-studio\node_modules\webpack-sources\lib\OriginalSource.js:64:32)
    at module.exports (C:\apollo-studio\apollo-studio\node_modules\webpack-sources\lib\helpers\streamChunks.js:13:17)
    at ReplaceSource.streamChunks (C:\apollo-studio\apollo-studio\node_modules\webpack-sources\lib\ReplaceSource.js:175:44)
    at module.exports (C:\apollo-studio\apollo-studio\node_modules\webpack-sources\lib\helpers\streamChunks.js:13:17)
    at ConcatSource.streamChunks (C:\apollo-studio\apollo-studio\node_modules\webpack-sources\lib\ConcatSource.js:112:55)
    at module.exports (C:\apollo-studio\apollo-studio\node_modules\webpack-sources\lib\helpers\streamChunks.js:13:17)
    at streamAndGetSourceAndMap (C:\apollo-studio\apollo-studio\node_modules\webpack-sources\lib\helpers\streamAndGetSourceAndMap.js:27:53)
    at CachedSource.streamChunks (C:\apollo-studio\apollo-studio\node_modules\webpack-sources\lib\CachedSource.js:208:35)
    at module.exports (C:\apollo-studio\apollo-studio\node_modules\webpack-sources\lib\helpers\streamChunks.js:13:17)

[alexander-akait]

Can you show this._value here https://github.com/webpack/webpack-sources/blob/main/lib/OriginalSource.js#L64?

[gardengeek99]

this._value.txt It hits it 321 times before failing with the exception.

[alexander-akait]

@gardengeek99 Can you try to disable source map and check again?

[gardengeek99]

It is already disabled in production builds (where the issue shows up). We set it to undefined, but explicitly setting it to false doesn't work either.

[alexander-akait]

@gardengeek99 so problems existing without source maps too?

[thre3eye]

I am facing the same issue. I can run a build with sourceMap=false but get above failure stack trace with sourceMap=true. This happens 100% of the time. (I guess there isn't any workaround to be able to debug without sourceMaps?)

-- @angular-devkit/build-angular@13.0.2 -- webpack@5.60.0 `-- webpack-sources@3.2.1