Closed andrewhathaway closed 6 years ago
I'm having this too with webpack-dev-server on Debian; but I haven't had time to debug this further. Do you have a minimal webpack config where you can reproduce this? It could also be an external loader or plugin.
Don't use hashes when watching. i. e. [chunkhash]
causes all assets to be stored.
@sokra In the documentation for css-loader it says to use hash for easier debugging wrt css modules. Does that have the same problem?
You can configure the generated ident with the
localIdentName
query parameter (default[hash:base64]
). Example:css-loader?localIdentName=[path][name]---[local]---[hash:base64:5]
for easier debugging.
We've also started to see this problem in dev (out of memory), but our only hash in dev mode is the css one.
EDIT: Of course it uses hash by default, so I'm guessing it's not a problem.
Does that have the same problem?
No. Only filenames.
You can compare it with webpack --watch
. If this generates many files in to your output path, the webpack-dev-server generates many files in the memory-fs.
If this is not the issue, you can increase the node.js memory (it defaults to 1.7 GB, which can be too few for big builds).
I'll test at work on Monday! If the outputted file is of the same name as the old one, then it's overwritten, right?
jup.
You can navigate to http://localhost:8080/webpack-dev-server to see a list of all files in memory.
Ah, didn't know that! Using a small project I have on my own computer, I notice that all of the hot-update files gets added.
Could they be a problem? Any way of clearing them out after they've been there for x amount of time? If we develop for a few hours there's probably quite a few of them in the end
Could they be a problem?
They could, but they are usually pretty small.
I think clearing them out like @SimenB said would be a nice option, or maybe there is some way to delete all in-memory files that are no longer in use?
jup. Write a plugin...
But deleting them too early can break stuff...
We have the exact same memory leak problem after using webpack -w on MacOS X. Can you please suggest what is a the best way to fix / workaround this issue. The ngOfficeUIFabric seems to deal with a different problem.
Thanks, Deepak
Have same issue of @deepakmani in OSX El Capitan, however i'm using Ionic 2 which in it's config doesn't use css-loader
, instead it just uses ionic-gulp-sass-build
, so ikd if the error happens because of webpack or that gulp plugin, also i'm not using HRM:
const path = require('path')
const PrefetchPlugin = require('webpack/lib/PrefetchPlugin')
const ProvidePlugin = require('webpack/lib/ProvidePlugin')
const DedupePlugin = require('webpack/lib/optimize/DedupePlugin')
const UglifyJsPlugin = require('webpack/lib/optimize/UglifyJsPlugin')
const WebpackNotifierPlugin = require('webpack-notifier')
module.exports = {
entry: {
polyfills: './app/polyfills.ts',
vendor: './app/vendor.ts',
app: path.resolve('app/app')
},
output: {
path: path.resolve('www/build/js'),
filename: '[name].bundle.js',
publicPath: "/",
chunkFilename: "[id].bundle.js",
pathinfo: false, // show module paths in the bundle, handy for debugging
},
devtool: 'eval',
module: {
loaders: [
{
test: /\.ts$/,
loader: 'awesome-typescript',
query: {
doTypeCheck: true,
resolveGlobs: false,
externals: ['typings/main.d.ts']
},
include: path.resolve('app'),
exclude: /node_modules/
}
,{
test: /\.js$/,
include: path.resolve('node_modules/angular2'),
loader: 'strip-sourcemap'
}
],
noParse: [
/es6-shim/,
/reflect-metadata/,
/zone\.js(\/|\\)dist(\/|\\)zone/
]
},
plugins: [
new PrefetchPlugin('rxjs'),
new ProvidePlugin({
Chart: 'chart.js/Chart.min.js'
}),
new DedupePlugin()
// ,new UglifyJsPlugin({
// // to debug prod builds uncomment //debug lines and comment //prod lines
// // beautify: true,//debug
// // mangle: false,//debug
// // dead_code: false,//debug
// // unused: false,//debug
// // deadCode: false,//debug
// // compress : { screw_ie8 : true, keep_fnames: true, drop_debugger: false, dead_code: false, unused: false, }, // debug
// // comments: true,//debug
// beautify: false,//prod
// // disable mangling because of a bug in angular2 beta.1, beta.2 and beta.3
// // TODO(mastertinner): enable mangling as soon as angular2 beta.4 is out
// // mangle: { screw_ie8 : true },//prod
// mangle: false,
// // dead_code : true,
// compress : {
// dead_code : true,
// screw_ie8 : true
// },//prod
// comments: false//prod
// })
,new WebpackNotifierPlugin({
title: 'tao_app_ionic',
alwaysNotify: true
// contentImage: path.join(appDir, 'images/notifier.png')
})
],
resolve: {
alias: {
'angular2': path.resolve('node_modules/angular2')
},
extensions: ['', '.js', '.ts']
}
};
I am having the same issue, any hack that can fix this for now.
Having the same problem running webpack
via webpack-stream
in gulp
. I use gulp.watch
to rerun on code changes, which then just runs
return gulp.src(tsStartFile)
.pipe(webpackStream(webPackConfig))
.on('error', logError)
.pipe(gulp.dest(outputDir))
I'd have thought this would run webpack from scratch each time, but after 5 or so builds I get the out of memory error and have to rerun my gulp file.
Did someone resolved this problem?
Another data point here; I have filename hashes disabled and my build isn't terribly large (it uses Angular 2, but few other dependencies), but after about an hour of dev work the watch dies with Abort trap: 6
.
I've temporarily increased the memory limit, which gives me a lot more time, but I'd rather not be consuming gigabytes just to run a watch command.
FYI if anyone is curious about upping the memory limit, I did it like so in my package.json (obviously the additional flags are dependent on your needs):
{
"scripts": {
"start": "node --max_old_space_size=4096 node_modules/webpack-dev-server/bin/webpack-dev-server.js --inline --progress --port 3000 --open"
}
}
Having the same issue as @thunderkid here. Any update?
Same issue for a very small project
For reference, using the flag isn't really helping me anymore and I think it was just a placebo. I don't think it's an actual memory issue because I've monitored system-level memory usage and I have yet to see it pass ~500MB.
I haven't found any way to get node to emit any additional error details beyond the abort trap. It sounds like maybe we should be escalating this with NodeJS team, since having an uninformative but consistent failure message is certainly a bug...
This just happened recently with a large project. It hadn't happened in the past. We did, however, just switch to using Docker.
I'm getting the FATAL ERROR: CALL_AND_RETRY_LAST Allocation failed - JavaScript heap out of memory
basically after every 5-10 rebuilds in dev mode using --watch. Is there any way to get around this in webpack 1.14.0?
Hello Webpack, after a few weeks of having webpack open and running I also got a memory error with "last resort gc" ... "Allocation failed - JavaScript heap out of memory Exit with" ... "[bg] Exited with code 3"
Same issue has just started for me. Has there been any update on this?
Also having this issue :( hashes for css-modules are a must for development!
Same problem. Our production builds have recently started failing too. Typescript + React. We've increased memory for our node processes to no avail. Any advice or what version to roll back to?
Same issue here. Running webpack dev server takes up ~3-3.5gb of ram on my machine
Security context: 0x354f7b2a66a1 <JS Object>
0: builtin exit frame: stringify(this=0x354f7b29a7d1 <a JSON with map 0x8ddf39053d9>,0xd2259a02311 <undefined>,0xd2259a02311 <undefined>,0x225fe8c83c11 <an Object with map 0x1460ed77b201>)
1: arguments adaptor frame: 1->3
2: /* anonymous */(aka /* anonymous */) [/Users/x/WebApp/node_modules/webpack/lib/SourceMapDevToolPlugin.js:~112] [pc=0xaf6d181150b](this=0xd2259a0231...
FATAL ERROR: CALL_AND_RETRY_LAST Allocation failed - JavaScript heap out of memory
I'm also getting this error, except in Webpack 2: 😞 (with webpack-dev-middleware)
webpack: Compiling...
webpack building...
webpack: wait until bundle finished: /index.html
<--- Last few GCs --->
107742 ms: Mark-sweep 1363.5 (1415.9) -> 1361.9 (1415.9) MB, 1137.0 / 0.0 ms [allocation failure] [GC in old space requested].
109016 ms: Mark-sweep 1361.9 (1415.9) -> 1361.9 (1415.9) MB, 1274.0 / 0.0 ms [allocation failure] [GC in old space requested].
110267 ms: Mark-sweep 1361.9 (1415.9) -> 1366.0 (1404.9) MB, 1251.0 / 0.0 ms [last resort gc].
111552 ms: Mark-sweep 1366.0 (1404.9) -> 1370.0 (1404.9) MB, 1285.0 / 0.0 ms [last resort gc].
<--- JS stacktrace --->
==== JS stack trace =========================================
Security context: 00000248DD2CFB61 <JS Object>
1: set [native collection.js:~252] [pc=000003C901E0A3E4] (this=000002BF0455F139 <a Map with map 0000015D1F80A191>,r=000003CA712E4119 <String[12]: linkParentID>,z=0000009BCCF0CBD9 <a TypeObject with map 000000F54E3505A1>)
2: getLiteralTypeForText [C:\Root\Apps\@V\DebateMap\Main\node_modules\typescript\lib\typescript.js:~30635] [pc=000003C900E6C15B] (this=00000248DD2E6661 <JS Global Obj...
FATAL ERROR: CALL_AND_RETRY_LAST Allocation failed - JavaScript heap out of memory
[nodemon] app crashed - waiting for file changes before starting...
EDIT
Okay, in my case, the problem is that I had:
FileA.tsx:
import {ButtonProps} from "../../../../Frame/ReactComponents/Button";
Button.tsx:
export type ButtonProps = {enabled?: boolean, [...]} & React.HTMLProps<HTMLDivElement>;
In other words, importing a type definition that depends on React's HTMLProps<HTMLDivElement>
type.
Interestingly, I made the same import of the ButtonProps
type from a different file, and it doesn't cause the build to fail there.
So perhaps it's a bug, or perhaps it's something else, like the fact that React's types use generics a lot.
Still doesn't fully explain it, though, since it shouldn't just be a memory limit being hit. (I upped the limit using --max_old_space_size=4096
, and it didn't change anything -- and the process only ever went up to 1.42gb of usage)
I'm experiencing the same issue here. From what can I tell, it seems that Webpack is loading up files in the project directory even when they are technically "unreachable" from the dependency graph, starting from the given entry points.
I'm trying to integrate Webpack into an existing project with a large existing Javascript code (written as Typescript). This is where Webpack throws OOM errors.
So as an experiment, I try to make a minimal Webpack test example, as described in this gist. This compiles fine, so far so good.
But, when I copy the src
directory of the large project into my minimal example, Webpack then throws OOM errors. Note that I just copied the src
directory; none of the files in the given gist example were changed.
FWIW, I ran Webpack with strace
and I get traces like
open("/node_modules/@types/zeroclipboard.d.ts", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
open("/node_modules/@types/zeroclipboard/package.json", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
open("/node_modules/@types/zeroclipboard/index.d.ts", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
...
open("/home/node_modules/zeroclipboard.js", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
open("/home/node_modules/zeroclipboard.jsx", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
open("/home/node_modules/zeroclipboard/package.json", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
open("/home/node_modules/zeroclipboard/index.js", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
open("/home/node_modules/zeroclipboard/index.jsx", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
I'm not sure if this is normal behavior or not but it is looking for node_modules
way beyond where it is supposed to look.
strace
also makes me sure that Webpack is indeed digging into the unreferenced src
directory. I get traces like,
open("/home/chad/kode/test/src/DefinitelyTyped/zeroclipboard/node_modules/zeroclipboard.tsx", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
open("/home/chad/kode/test/src/DefinitelyTyped/zeroclipboard/node_modules/zeroclipboard.d.ts", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
open("/home/chad/kode/test/src/DefinitelyTyped/zeroclipboard/node_modules/zeroclipboard/package.json", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
open("/home/chad/kode/test/src/DefinitelyTyped/zeroclipboard/node_modules/zeroclipboard/index.ts", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
open("/home/chad/kode/test/src/DefinitelyTyped/zeroclipboard/node_modules/zeroclipboard/index.tsx", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
open("/home/chad/kode/test/src/DefinitelyTyped/zeroclipboard/node_modules/zeroclipboard/index.d.ts", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
open("/home/chad/kode/test/src/DefinitelyTyped/zeroclipboard/node_modules/@types/zeroclipboard.d.ts", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
open("/home/chad/kode/test/src/DefinitelyTyped/zeroclipboard/node_modules/@types/zeroclipboard/package.json", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
open("/home/chad/kode/test/src/DefinitelyTyped/zeroclipboard/node_modules/@types/zeroclipboard/index.d.ts", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
Three hours? I think the problem is just that you need more patience.
+1 same issue here deploying to heroku
Not sure if there is anyone else here using webpacker, but the problem for me was that in config/webpack/shared.js
they changed this:
const extensionGlob = `*{${paths.extensions.join(',')}}*`
to this:
const extensionGlob = `**/*{${settings.extensions.join(',')}}*`
Which explains why it may be traversing down directories and loading files that are never actually used by the dependency graph. Dropping the **/
fixed the out of memory issue for me.
The problem for me was a memory leak in postcss-loader v1.3.x, no more crashes so far after an upgrade to v.2.0.5
@ertrzyiks thanks for the info. Do you have a link referencing the memory leak (i.e. a Github PR or Issue) or is that something you discovered on your own?
@dcalhoun it's something I discovered on my own, I couldn't find any commit dedicated to fixing this, a memory leak has been removed as a side effect of recent refactoring I guess
@eirikurn Alright. After looking into it, it looks like postcss-loader#2.0.0 did update to postcss#6.0, which appears to include some memory related improvements.
Not sure if this change is what impacted your improvement, but it's at least good to know. Thanks again.
We faced a similar problem with webpack-dev-server. Using a more aggressive policy for CommonsChunk plugin reduced memory usage from 1.8GB to 1.1GB, minChunks
was set to 3 to include most of the shared code:
new webpack.optimize.CommonsChunkPlugin({
name: 'common',
filename: 'common.js',
minChunks: 3,
}),
Is there a way to disable the usage of memory of FS and to use regular file system?
My situation may be a bit specific, but I had a few very large .d.ts
files in the project that caused the webpack build to fail all the time (out of memory). Adding skipLibCheck: true
to tsconfig.json
fixed that.
I had same issues, i set --max_old_space_size=4096
in webpack.cmd & gulp.cmd,but no effect,this issues only happen in production,maybe because use hash file name.
additional: I used webpack with gulp,it has three webpack task in gulpfile,if i manual run task standalone,all work fine,but run in gulp task,the memory sustained growth,it seems not release memory after one task end.
I have same issue, though I am using ionic app scripts, with webpack 3.0 the memory spikes up to 5gb and the process dies. With webpack 2 it used to be ok
Just upgraded to webpack 3.0 and experiencing leaks as well:
16:17:03 - File change detected. Starting incremental compilation...
<--- Last few GCs --->
[26089:0x3d30680] 47717 ms: Mark-sweep 1402.9 (1470.8) -> 1400.8 (1472.8) MB, 891.8 / 0.0 ms allocation failure GC in old space requested
[26089:0x3d30680] 48960 ms: Mark-sweep 1400.8 (1472.8) -> 1400.8 (1472.8) MB, 1243.3 / 0.0 ms allocation failure GC in old space requested
[26089:0x3d30680] 50225 ms: Mark-sweep 1400.8 (1472.8) -> 1400.8 (1442.8) MB, 1264.0 / 0.0 ms last resort
[26089:0x3d30680] 51476 ms: Mark-sweep 1400.8 (1442.8) -> 1400.8 (1442.8) MB, 1251.0 / 0.0 ms last resort
<--- JS stacktrace --->
==== JS stack trace =========================================
Security context: 0x1c3674e29891 <JS Object>
2: encodeLastRecordedSourceMapSpan [/home/rjansen/CODE/core/node_modules/typescript/lib/tsc.js:~50611] [pc=0xdef7f5f96d8](this=0x264e5bcb27a1 <JS Global Object>)
3: emitPos [/home/rjansen/CODE/core/node_modules/typescript/lib/tsc.js:~50639] [pc=0xdef7f5525d1](this=0x264e5bcb27a1 <JS Global Object>,pos=0x2caa051c92e9 <Number: 5.37458e+06>)
4: emitNodeWithSourceMap [/home/rjansen/CODE...
FATAL ERROR: CALL_AND_RETRY_LAST Allocation failed - JavaScript heap out of memory
1: node::Abort() [node]
2: 0x13647ec [node]
3: v8::Utils::ReportOOMFailure(char const*, bool) [node]
4: v8::internal::V8::FatalProcessOutOfMemory(char const*, bool) [node]
5: v8::internal::Factory::NewUninitializedFixedArray(int) [node]
6: 0xe90ca3 [node]
7: v8::internal::Runtime_GrowArrayElements(int, v8::internal::Object**, v8::internal::Isolate*) [node]
8: 0xdef7de0437d
Aborted
npm ERR! code ELIFECYCLE
npm ERR! errno 134
npm ERR! core@1.0.0 watch-server: `tsc -p tsconfig.json -inlineSourceMap -outDir private --watch`
npm ERR! Exit status 134
After every 5-6 build on dev mode including--hot --inline --progress --colors --port 2000
,
i got this FATAL ERROR: CALL_AND_RETRY_LAST Allocation failed - JavaScript heap out of memory
webpack
v3.1.0
webpack-dev-server
v2.5.1
I have same issues, when I build with npm run clean && webpack --display-error-details, I got this:
<--- Last few GCs --->
164635 ms: Mark-sweep 1275.2 (1434.0) -> 1275.2 (1434.0) MB, 1446.8 / 0.0 ms [allocation failure] [GC in old space requested].
166101 ms: Mark-sweep 1275.2 (1434.0) -> 1275.2 (1434.0) MB, 1465.7 / 0.0 ms [allocation failure] [GC in old space requested].
167625 ms: Mark-sweep 1275.2 (1434.0) -> 1280.3 (1413.0) MB, 1523.3 / 0.1 ms [last resort gc].
169164 ms: Mark-sweep 1280.3 (1413.0) -> 1285.7 (1413.0) MB, 1538.1 / 0.0 ms [last resort gc].
<--- JS stacktrace --->
==== JS stack trace =========================================
Security context: 00000168FC0CFB61 <JS Object>
1: map [native array.js:~994] [pc=000002493CE23EA3] (this=000003E091F1A8A1 <JS Array[1]>,bq=0000031BDD71D189 <JS Function (SharedFunctionInfo 000002FDA0B639D1)>,br=00000168FC004381 <undefi
ned>)
2: arguments adaptor frame: 1->2
3: new BasicSourceMapConsumer [C:\Users\a181226\Documents\cancelacion-webpack\node_modules\source-map\lib\source-map-consumer.js:314] [pc=000002493EBB3B52] ...
FATAL ERROR: CALL_AND_RETRY_LAST Allocation failed - JavaScript heap out of memory
I use webpack 2.6.1
Same issue
<--- Last few GCs --->
7076693 ms: Scavenge 1416.9 (1472.9) -> 1416.9 (1472.9) MB, 3.9 / 0 ms (+ 5.4 ms in 1 steps since last GC) [allocation failure] [incremental marking delaying mark-sweep].
7077971 ms: Mark-sweep 1416.9 (1472.9) -> 1403.9 (1459.8) MB, 1278.4 / 0 ms (+ 9.1 ms in 2 steps since start of marking, biggest step 5.4 ms) [last resort gc].
7079257 ms: Mark-sweep 1403.9 (1459.8) -> 1403.9 (1459.8) MB, 1285.9 / 0 ms [last resort gc].
<--- JS stacktrace --->
==== JS stack trace =========================================
Security context: 0x2fb44c1e3ac1 <JS Object>
2: toString [/home/abarrera/Proyectos/front-sae/node_modules/webpack/lib/Stats.js:411] [pc=0x24c131efa4ef] (this=0x3546c1922b51 <a Stats with map 0x38d80a68ee9>,options=0x2513409b01f9 <an Object with map 0x1ad3a812ccc1>)
3: handler(aka compilerCallback) [/home/abarrera/Proyectos/front-sae/node_modules/webpack/bin/webpack.js:348] [pc=0x24c131efa08d] (this=0x2513409b3071 <a Watching with ...
FATAL ERROR: CALL_AND_RETRY_LAST Allocation failed - process out of memory
Command used
webpack --watch --progress
Webpack
v3.2.0
Node
v5.12.0
Having this problem as well. Tried couple of things like disabling uglification for a while, to no avail. It gets stuck 92% chunk asset optimization
for a while and then throws that FATAL ERROR: CALL_AND_RETRY_LAST Allocation failed - JavaScript heap out of memory
. Tried using https://www.npmjs.com/package/increase-memory-limit didn't get either. I don't know anymore.
Also getting this, OOM error:
[28798:0x3a4d6c0] 533443 ms: Mark-sweep 1379.0 (1544.1) -> 1378.9 (1543.1) MB, 439.8 / 0.0 ms allocation failure GC in old space requested
[28798:0x3a4d6c0] 533926 ms: Mark-sweep 1378.9 (1543.1) -> 1378.9 (1497.6) MB, 482.9 / 0.0 ms last resort
[28798:0x3a4d6c0] 534372 ms: Mark-sweep 1378.9 (1497.6) -> 1378.7 (1484.6) MB, 445.6 / 0.1 ms last resort
<--- JS stacktrace --->
==== JS stack trace =========================================
Security context: 0x130e83a266a1 <JS Object>
1: DoJoin(aka DoJoin) [native array.js:~97] [pc=0x11ae132ebe76](this=0x147598602311 <undefined>,q=0x146650379c31 <JS Array[2]>,r=2,F=0x1475986023b1 <true>,B=0x147598602471 <String[0]: >,A=0x147598602421 <false>)
2: Join(aka Join) [native array.js:~122] [pc=0x11ae130f8622](this=0x147598602311 <undefined>,q=0x146650379c31 <JS Array[2]>,r=2,B=0x147598602471 <String[0]: >,A=0x147598602421 <...
FATAL ERROR: CALL_AND_RETRY_LAST Allocation failed - JavaScript heap out of memory
1: node::Abort() [node]
2: 0x12b82ac [node]
3: v8::Utils::ReportOOMFailure(char const*, bool) [node]
4: v8::internal::V8::FatalProcessOutOfMemory(char const*, bool) [node]
5: v8::internal::Factory::NewRawTwoByteString(int, v8::internal::PretenureFlag) [node]
6: v8::internal::Runtime_StringBuilderConcat(int, v8::internal::Object**, v8::internal::Isolate*) [node]
7: 0x11ae0fc843a7
Aborted
Seems related to Webpack 3 upgrade for me as well
On Node v8.1.2 with upgrade from webpack@2.4.1
=> webpack@3.0.0
caused much slowdown and the CALL_AND_RETRY_LAST Allocation failed - JavaScript heap out of memory
error.
Upon doing:
rm -rf node_modules
and then npm install
with webpack@2.4.1
, the problem goes away, and compiling/build times are dramatically reduced.
My error is directly related to: #5089
@stevelacy same here man. Have an issue https://github.com/webpack/webpack/issues/5257
After a couple of builds running in a Watch, Webpack crashes due to being out of memory.