angular / angular-cli

CLI tool for Angular
https://cli.angular.dev
MIT License
26.79k stars 11.98k forks source link

AOT build fails due to "JavaScript heap out of memory" #5618

Closed ghost closed 5 years ago

ghost commented 7 years ago

Bug Report or Feature Request (mark with an x)

- [X ] bug report -> please search issues before submitting
- [ ] feature request

Versions.

@angular/cli: 1.0.0
node: 7.7.4
os: win32 x64
@angular/animations: 4.0.0
@angular/common: 4.0.0
@angular/compiler: 4.0.0
@angular/core: 4.0.0
@angular/flex-layout: 2.0.0-rc.1
@angular/forms: 4.0.0
@angular/http: 4.0.0
@angular/material: 2.0.0-beta.2
@angular/platform-browser: 4.0.0
@angular/platform-browser-dynamic: 4.0.0
@angular/router: 4.0.0
@angular/cli: 1.0.0
@angular/compiler-cli: 4.0.0

Repro steps.

ng build --prod --aot

The log given by the failure.

/node_modules/@angular/material/aut 92% chunk asset optimization
<--- Last few GCs --->

[6328:0000016AB4D09F00]  1775597 ms: Mark-sweep 1411.2 (1513.5) -> 1411.0 (1513.5) MB, 1240.6 / 0.0 ms  last resort

<--- JS stacktrace --->

==== JS stack trace =========================================

Security context: 0000034B0DD266A1 <JS Object>
    2: optimize [0000000FABC02311 <undefined>:5480] [pc=00000082575652C8](this=000000633199A279 <an AST_Call with map 0000014753FA0291>,compressor=00000226DBDECA61 <a Compressor with map 0000032B1EC8F829>)
    3: before [0000000FABC02311 <undefined>:5463] [pc=0000008258C1818D](this=00000226DBDECA61 <a Compressor with map 0000032B1EC8F829>,node=000000633199A279 <an AST_Call with map 0000014753...

FATAL ERROR: CALL_AND_RETRY_LAST Allocation failed - JavaScript heap out of memory

Desired functionality.

I would like to be able the app using AOT.

Mention any other details that might be useful.

I have tried changing ng.cmd to this:

@IF EXIST "%~dp0\node.exe" (
  "%~dp0\node.exe" --max_old_space_size=8192 "%~dp0\..\@angular\cli\bin\ng" %*
) ELSE (
  @SETLOCAL
  @SET PATHEXT=%PATHEXT:;.JS;=;%
  node --max_old_space_size=8192 "%~dp0\..\@angular\cli\bin\ng" %*
)

and ngc.cmd to:

@IF EXIST "%~dp0\node.exe" (
  "%~dp0\node.exe" --max_old_space_size=8192 "%~dp0\..\@angular\compiler-cli\src\main.js" %*
) ELSE (
  @SETLOCAL
  @SET PATHEXT=%PATHEXT:;.JS;=;%
  node --max_old_space_size=8192 "%~dp0\..\@angular\compiler-cli\src\main.js" %*
)

I have 8GB swap file. The build fails after 20-25 minutes standing on 92% progress.

filipesilva commented 7 years ago

This is similar to https://github.com/angular/angular-cli/issues/1652, but seems localized to AOT compilations. There might be improvements we can do to help.

iuristona commented 7 years ago

I got the same issue too.

So, I tried to build the same project (@angular 4.0.0) using ng-cli 1.0.0-rc.4 and it works fine. ng build --prod --aot

@angular/cli: 1.0.0-rc.4 node: 6.9.5 os: win32 x64 @angular/common: 4.0.0 @angular/compiler: 4.0.0 @angular/core: 4.0.0 @angular/forms: 4.0.0 @angular/http: 4.0.0 @angular/platform-browser: 4.0.0 @angular/platform-browser-dynamic: 4.0.0 @angular/router: 4.0.0 @angular/cli: 1.0.0-rc.4 @angular/compiler-cli: 4.0.0

ghost commented 7 years ago

I also want to add that even though I try to increase max_old_space_size by editing ng.cmd and ngc.cmd, it seems like the node.js process never uses more than 1600 MB of RAM according to the process monitor on my system, which is well within the limit i setup of 8192 MB and it still says "out of memory".

tonny008 commented 7 years ago

I have the same issue. my project works on cli rc.4. it fails when ng build --prod after update to 1.0.0.

aminebizid commented 7 years ago

I'm so afraid there will be no solution for this issue. I reverted to rc.2 because of this :(

efstathiosntonas commented 7 years ago

In my case, I installed typescript@2.1.6 instead of @~2.2.0 and the memory issue went away. Give it a try.

There is an issue in ~2.2.0 and @angular/language-service 4.0.0 (in my case i use it)

AngelVlc commented 7 years ago

I've changed max_old_space_size in %AppData%\npm (Windows) and it works for me when launching ng serve with 1.0.0.

aminebizid commented 7 years ago

@efstahiosntonas are you using ng4?

efstathiosntonas commented 7 years ago

@zigzag95 yes, ng4

elvisbegovic commented 7 years ago

same problem here

ghost commented 7 years ago

Thanks @AngelVlc !

Changing limit in ng.cmd under %appdata% did the trick! Now node.js uses around 4GB of RAM and the build is completed.

chrisnicola commented 7 years ago

I'm getting OOM errors with @ngtools/webpack 1.3.0 and @angular 4.0.0. This is on OSX and not Windows as well.

<--- Last few GCs --->

   17671 ms: Mark-sweep 1241.3 (1413.6) -> 1241.3 (1413.6) MB, 615.1 / 0.0 ms [allocation failure] [GC in old space requested].
   18286 ms: Mark-sweep 1241.3 (1413.6) -> 1241.3 (1413.6) MB, 614.2 / 0.0 ms [allocation failure] [GC in old space requested].
   18897 ms: Mark-sweep 1241.3 (1413.6) -> 1249.0 (1410.6) MB, 610.8 / 0.0 ms [last resort gc].
   19484 ms: Mark-sweep 1249.0 (1410.6) -> 1256.6 (1410.6) MB, 586.6 / 0.0 ms [last resort gc].

<--- JS stacktrace --->

==== JS stack trace =========================================

Security context: 0x51588dcfb51 <JS Object>
    1: /* anonymous */ [/Users/chrisnicola/src/wealthbar/node_modules/webpack/node_modules/enhanced-resolve/lib/UnsafeCachePlugin.js:~28] [pc=0x33d385eb70d4] (this=0xd76ce5e1e71 <a Resolver with map 0x20b5bd9fe3f1>,request=0x3f6f1fa5ec39 <an Object with map 0x1fbc4c2b91c1>,callback=0xfd0ab9f8a9 <JS Function (SharedFunctionInfo 0x395f78d2a061)>)
    2: applyPluginsParallelBailResult1 [/Users/chri...

FATAL ERROR: CALL_AND_RETRY_LAST Allocation failed - JavaScript heap out of memory
 1: node::Abort() [/usr/local/Cellar/node/6.9.1/bin/node]
 2: node::FatalException(v8::Isolate*, v8::Local<v8::Value>, v8::Local<v8::Message>) [/usr/local/Cellar/node/6.9.1/bin/node]
 3: v8::Utils::ReportApiFailure(char const*, char const*) [/usr/local/Cellar/node/6.9.1/bin/node]
 4: v8::Utils::ApiCheck(bool, char const*, char const*) [/usr/local/Cellar/node/6.9.1/bin/node]
 5: v8::internal::V8::FatalProcessOutOfMemory(char const*, bool) [/usr/local/Cellar/node/6.9.1/bin/node]
 6: v8::internal::Factory::NewInternalizedStringImpl(v8::internal::Handle<v8::internal::String>, int, unsigned int) [/usr/local/Cellar/node/6.9.1/bin/node]
 7: v8::internal::InternalizedStringKey::AsHandle(v8::internal::Isolate*) [/usr/local/Cellar/node/6.9.1/bin/node]
 8: v8::internal::StringTable::LookupKey(v8::internal::Isolate*, v8::internal::HashTableKey*) [/usr/local/Cellar/node/6.9.1/bin/node]
 9: v8::internal::StringTable::LookupString(v8::internal::Isolate*, v8::internal::Handle<v8::internal::String>) [/usr/local/Cellar/node/6.9.1/bin/node]
10: v8::internal::LookupIterator::LookupIterator(v8::internal::Handle<v8::internal::Object>, v8::internal::Handle<v8::internal::Name>, v8::internal::LookupIterator::Configuration) [/usr/local/Cellar/node/6.9.1/bin/node]
11: v8::internal::LookupIterator::PropertyOrElement(v8::internal::Isolate*, v8::internal::Handle<v8::internal::Object>,v8::internal::Handle<v8::internal::Object>, bool*, v8::internal::LookupIterator::Configuration) [/usr/local/Cellar/node/6.9.1/bin/node]
12: v8::internal::Runtime::GetObjectProperty(v8::internal::Isolate*, v8::internal::Handle<v8::internal::Object>, v8::internal::Handle<v8::internal::Object>) [/usr/local/Cellar/node/6.9.1/bin/node]
13: v8::internal::Runtime_KeyedGetProperty(int, v8::internal::Object**, v8::internal::Isolate*) [/usr/local/Cellar/node/6.9.1/bin/node]
14: 0x33d384d092a7
[1]    9536 abort      npm start
marcalj commented 7 years ago

Same problem here with:

@angular/cli: 1.0.0-rc.2 node: 6.9.2 os: darwin x64 @angular/common: 2.4.10 @angular/compiler: 2.4.10 @angular/core: 2.4.10 @angular/forms: 2.4.10 @angular/http: 2.4.10 @angular/platform-browser: 2.4.10 @angular/platform-browser-dynamic: 2.4.10 @angular/router: 3.4.10 @angular/cli: 1.0.0-rc.2 @angular/compiler-cli: 2.4.10

Ristaaf commented 7 years ago

I have the same problem when running "ng build -w" @angular/cli: 1.0.0 node: 7.8.0 @angular/* 4.0.2

setting --max_old_space_size=8192 works for me, but it is not really a good solution since I cannot redistribute it with the source code to my developers.

One note though, there are both a ng.cmd and an ng file (the latter one is an sh file), and if you are in windows as we are both can be used depending on if you run ng from git bash or cmd so both needs to be changed.

My solution to the problem will probably be to change the npm build command from ng build to call a copy of the ng bash file which I distribute with my code.

omeralper commented 7 years ago

@Ristaaf How did you change the ng file? Where did you add --max_old_space_size in this file? Can you please share your file.

Ristaaf commented 7 years ago

@omeralper: This is my solution, it requires people to use git bash as terminal if on windows, but it would be easy to change if needed (just use the cmd file instead):

In my project root I have a folder called scripts and in it a file called ng.sh, which is a copy from node_modules/.bin/ng but with more allowed RAM to be used

#!/bin/sh
basedir=$(dirname "$(echo "$0" | sed -e 's,\\,/,g')")

case `uname` in
    *CYGWIN*) basedir=`cygpath -w "$basedir"`;;
esac

if [ -x "$basedir/node" ]; then
  "$basedir/node" --max_old_space_size=8192 "./node_modules/@angular/cli/bin/ng" "$@"
  ret=$?
else
  node --max_old_space_size=8192 "./node_modules/@angular/cli/bin/ng" "$@"
  ret=$?
fi
exit $ret

Then in my package.json I do:

"scripts": {
    "build-prod": "bash ./scripts/ng.sh build --prod --aot --env=prod"
}
KeithGillette commented 7 years ago

I am experiencing a similar issue resulting in a JavaScript heap out of memory error when calling ng e2e in a project using:

I was able to create a cross-platform work-around by changing the package.json e2e script entry to:

    "test-browser-e2e-cucumber": "node --max_old_space_size=4096 node_modules/@angular/cli/bin/ng e2e --webdriver-update",
speedy32 commented 7 years ago

On windows @IF EXIST "%~dp0\node.exe" ( "%~dp0\node.exe" --max_old_space_size=4096 "%~dp0\node_modules\@angular\cli\bin\ng" % ) ELSE ( @SETLOCAL @SET PATHEXT=%PATHEXT:;.JS;=;% node --max_old_space_size=4096 "%~dp0\node_modules\@angular\cli\bin\ng" % )

dakotamurphyucf commented 7 years ago

check this npm package out https://github.com/endel/increase-memory-limit

isaacplmann commented 7 years ago

I wrote a little script that lets me control the memory limit in the config section of package.json. https://github.com/isaacplmann/angular-cli-alias

increase-memory-limit looks pretty sweet too. It didn't exist when I wrote my script and it doesn't let you set your own memory limit.

zackschuster commented 7 years ago

i've been able to repro this with a minimal test case using an out-of-the-box cli app; relevant file here.

basically it seems like trying to match a wildcard without specifiers will blow up the app, which is weird since the test case is pulled almost directly from typescript documentation.

note that "*": ["*"] does not crash the app, only "*": ["*", "$dir/*"]

as a workaround, i've manually specified each of my path maps, which is annoying but lets me compile:

"paths": {
  "app.*": ["app/app.*"],
  "app-*": ["app/app-*"],
  "authentication/*": ["app/authentication/*"],
  "core/*": ["app/core/*"],
  "navigation/*": ["app/navigation/*"],
  "pages/*": ["app/pages/*"]
}
sis0k0 commented 7 years ago

We also hit that issue in NativeScript when using the @ngtools/webpack plugin with nativescript-dev-webpack. We use tsconfig paths for mapping our tns-core-modules. They look like that:

"paths": {
  "*": [
    "./node_modules/*",
    "./node_modules/tns-core-modules/*"
  ]
}

However, the bundling fails with the mentioned error (JavaScript heap out of memory). Just like @zackschuster , as a workaround we specified each path mapping manually:

"paths": {
  "application": ["node_modules/tns-core-modules/application"],
  "application-settings": ["node_modules/tns-core-modules/application-settings"],
  "camera": ["node_modules/tns-core-modules/camera"],
  "color": ["node_modules/tns-core-modules/color"],
  "connectivity": ["node_modules/tns-core-modules/connectivity"],
  "console": ["node_modules/tns-core-modules/console"],
  "data/*": ["node_modules/tns-core-modules/data/*"],
  "fetch": ["node_modules/tns-core-modules/fetch"],
  "file-system": ["node_modules/tns-core-modules/file-system"],
  "fps-meter": ["node_modules/tns-core-modules/fps-meter"],
  "globals": ["node_modules/tns-core-modules/globals"],
  "http": ["node_modules/tns-core-modules/http"],
  "image-asset": ["node_modules/tns-core-modules/image-asset"],
  "image-source": ["node_modules/tns-core-modules/image-source"],
  "location": ["node_modules/tns-core-modules/location"],
  "platform": ["node_modules/tns-core-modules/platform"],
  "text": ["node_modules/tns-core-modules/text"],
  "timer": ["node_modules/tns-core-modules/timer"],
  "trace": ["node_modules/tns-core-modules/trace"],
  "ui/*": ["node_modules/tns-core-modules/ui/*"],
  "utils/*": ["node_modules/tns-core-modules/utils/*"],
  "xhr": ["node_modules/tns-core-modules/xhr"],
  "xml": ["node_modules/tns-core-modules/xml"]
}
zackschuster commented 7 years ago

@sis0k0 what do your recompile times look like with that setup? i was getting 40-50s recompiles (on an absurdly tiny project) until i dropped the path maps, at which point it dropped to ~2s.

aminebizid commented 7 years ago

@zackschuster What do you mean by path maps?

zackschuster commented 7 years ago

@zigzag95

"paths": {
 // ...
}
sis0k0 commented 7 years ago

@zackschuster, similar times.

tgrosh commented 7 years ago

https://github.com/angular/angular-cli/issues/5618#issuecomment-304545043

I tried the increase-memory-limit package, and I still get Javascript heap out of memory

ghost commented 7 years ago

Still a problem on 4.2.6, macOS, even when giving 8GB to node.

This is a pretty serious problem... it's time to fix this.

elvisbegovic commented 7 years ago

It's time to fix this !!! In real big project angular is bad because of that ! Maybe webpack problem !! Thanks

ghost commented 7 years ago

I wonder if the solution is to AOT per module, then a separate link-step? Not sure if it's even possible to implement... In that case, what is lost in global opts, needs to be reclaimed in an LTO.

cassmtnr commented 7 years ago

Still nothing on this?

I am using @angular/cli 1.0.0-rc.4, if I upgrade to 1.0.0 it gives me the error of angular cli FATAL ERROR: CALL_AND_RETRY_LAST Allocation failed - JavaScript heap out of memory

aminebizid commented 7 years ago

Now we understand why Google don't use webpack with ng for its apps

cimachris commented 7 years ago

Work Around Fix I was able to fix the issue by changing my ng.cmd file to
@IF EXIST "%~dp0\node.exe" ( "%~dp0\node.exe" --max_old_space_size=8192 "%~dp0\node_modules\@angular\cli\bin\ng" %* ) ELSE ( @SETLOCAL @SET PATHEXT=%PATHEXT:;.JS;=;% node --max_old_space_size=8192 "%~dp0\node_modules\@angular\cli\bin\ng" %* )

Masquer commented 7 years ago

@cimachris this is not exactly a fix but rather a workaround that can possibly postpone the problem

cimachris commented 7 years ago

Is that better?

cassmtnr commented 7 years ago

@hansl @filipesilva I am not able to upgrade my application to angular/cli 1.0.0, since I am at angular/cli 1.0.0.rc-4, don't want to make the jump to the latest version without testing my components

ghost commented 7 years ago

@cimachris we're well beyond the "--max_old_space_size" trick working anymore. Once the app gets large enough, aot builds are impossible due to this problem. Granted, we're an extreme case, because we have lots and lots of generated components (based on screen-scraped definitions of a legacy system), but it should still build of course. This is a very serious bug IMO.

edmund-lee commented 7 years ago

Have been affected by this issue since @angular/cli 1.0 was released But currently we are able to workaround it by running ng build --aot --sourcemaps=false --environment=prod

It's not exactly the same as the --prod parameter but at least it works.

xnnkmd commented 7 years ago

ng e2e also results in a similar error:


 92% chunk asset optimization
<--- Last few GCs --->

[2164:000002C715EFDC90]   117246 ms: Mark-sweep 1401.4 (1543.7) -> 1401.3 (1545.7) MB, 1088.0 / 0.0 ms  allocation failure GC in old space requested
[2164:000002C715EFDC90]   118473 ms: Mark-sweep 1401.3 (1545.7) -> 1401.2 (1504.2) MB, 1227.0 / 0.0 ms  last resort
[2164:000002C715EFDC90]   119591 ms: Mark-sweep 1401.2 (1504.2) -> 1401.1 (1504.2) MB, 1117.8 / 0.0 ms  last resort

<--- JS stacktrace --->

==== JS stack trace =========================================

Security context: 000000799D9A66A1 <JS Object>
    2: addMapping(aka SourceMapGenerator_addMapping) [xxx\node_modules\source-map\lib\source-map-generator.js:~94] [pc=0000003D3772300C](this=0000022DE262CAF1 <a SourceMapGenerator with map 000000A8E17906F1>,aArgs=0000005D8F791BF9 <an Object with map 00000399007FF561>)
    3: /* anonymous */ [xxx\node_modules\source-map\l...

FATAL ERROR: CALL_AND_RETRY_LAST Allocation failed - JavaScript heap out of memory
xnnkmd commented 7 years ago

@filipesilva This is pretty serious and the workaround does not allways work - it seems to affect all medium/large projects. Needs solved for angular + angularcli to remain a viable technology for serious web projects.

ghost commented 7 years ago

@edmund-lee

ng build --aot --sourcemaps=false --environment=prod

It's not exactly the same as the --prod parameter but at least it works.

Doesn't work for sufficiently large projects.

emazv72 commented 7 years ago

@angular/cli: 1.2.3 node: 6.10.0 os: linux x64 @angular/animations: 4.3.1 @angular/common: 4.3.1 @angular/compiler: 4.3.1 @angular/core: 4.3.1 @angular/forms: 4.3.1 @angular/http: 4.3.1 @angular/platform-browser: 4.3.1 @angular/platform-browser-dynamic: 4.3.1 @angular/router: 4.3.1 @angular/cli: 1.2.3 @angular/compiler-cli: 4.3.1 @angular/language-service: 4.3.1

App with 3 modules and about 50 components, after switching @angular/cli from 1.1.x to 1.2.x

ng build -target=production

FATAL ERROR: CALL_AND_RETRY_LAST Allocation failed - JavaScript heap out of memory
 1: node::Abort() [@angular/cli]
 2: 0x109bf8c [@angular/cli]
 3: v8::Utils::ReportApiFailure(char const*, char const*) [@angular/cli]
 4: v8::internal::V8::FatalProcessOutOfMemory(char const*, bool) [@angular/cli]
 5: v8::internal::Factory::NewInternalizedStringImpl(v8::internal::Handle<v8::internal::String>, int, unsigned int) [@angular/cli]
 6: v8::internal::StringTable::LookupString(v8::internal::Isolate*, v8::internal::Handle<v8::internal::String>) [@angular/cli]
 7: v8::internal::LookupIterator::PropertyOrElement(v8::internal::Isolate*, v8::internal::Handle<v8::internal::Object>, v8::internal::Handle<v8::internal::Object>, bool*, v8::internal::LookupIterator::Configuration) [@angular/cli]
 8: v8::internal::Runtime::GetObjectProperty(v8::internal::Isolate*, v8::internal::Handle<v8::internal::Object>, v8::internal::Handle<v8::internal::Object>) [@angular/cli]
 9: 0xed3501 [@angular/cli]
10: v8::internal::Runtime_KeyedGetProperty(int, v8::internal::Object**, v8::internal::Isolate*) [@angular/cli]
11: 0x2174585092a7
Aborted (core dumped)
emazv72 commented 7 years ago

root cause analysis:

After fixing it:

ng build --prod

ERROR in ./src/main.ts Module not found: Error: Can't resolve './$$_gendir/app/app.module.ngfactory' @ ./src/main.ts 3:0-74 @ multi ./src/main.ts ` Upgraded @angular/cli to 1.2.4, still getting the same error ( yarn )

After looking at #7125 it looked like it was an aot related, package dependency issue. Reverting back to npm fixed it.

zackschuster commented 7 years ago

i've also seen this error pop up when using @ngtools/webpack by itself. interestingly, it doesn't happen until after a few rebuilds, implying there's a huge memory leak somewhere in either webpack (doubtful) or the ngtools plugin.

RuedigerMoeller commented 7 years ago

this is so java :))

baio commented 7 years ago

Same issue. As soon as project gain size this thing happens.

etiennetalbot commented 7 years ago

Exact same error here, but with @ngtools/webpack I've tried running it with Node LTS 6.11.2 and Node Latest as well 8.2.1 Tried with Typescript 2.3.2 and 2.4.2. All my angular packages are at 4.3.3

AoT is pretty much necessary for large apps, so it's kinda ironic we can't get it so build for those.

amikitevich commented 7 years ago

after update to angular/cli@1.3.0 have the same issue.

@angular/cli: 1.3.0 node: 8.2.1 os: darwin x64 @angular/animations: 4.3.3 @angular/common: 4.3.3 @angular/compiler: 4.3.3 @angular/core: 4.3.3 @angular/forms: 4.3.3 @angular/http: 4.3.3 @angular/platform-browser: 4.3.3 @angular/platform-browser-dynamic: 4.3.3 @angular/router: 4.3.3 @angular/cli: 1.3.0 @angular/compiler-cli: 4.3.3

command: ng build -prod --build-optimizer --aot=true --progress=false --sourcemap

brunobertechini commented 7 years ago

Just as a heads up. It turns out when I use YARN instead of NPM to RESTORE packages ng build completes successfully.

I am using

ng build --env=prod --aot

When I restore packages using npm install -- same error from this issue (out of memory) When I restore packages using yarn install -- no error - success

blyndcide commented 7 years ago

Also got this error after updating to @angular/cli 1.3.0.

Problem solved after upgrading typescript both locally and globally to 2.4.2