Node process for "Build" command is taking much more RAM (6GB) than configured "max_old_space_size" option (4GB)

sangalao commented 11 months ago

Command

build

Is this a regression?

[ ] Yes, this behavior used to work in the previous version

The previous version in which this bug was not present was

No response

Description

Hello,

The build command I run for my project is :

node --max_old_space_size=4096 node_modules/@angular/cli/bin/ng build --configuration production

This commands runs successfully, but however the NodeJS process for building app takes up to 6GB.

To be more precise, during beginning-middle of building phase it takes less than 4GB, and always at the middle-end of building phase, it goes up to 6GB just before end of successful build.

To start investigation, I launched build command with "-trace-gc" option :

node --trace-gc --max_old_space_size=4096 node_modules/@angular/cli/bin/ng build --configuration production

I saw, with this option, that there are several Node "V8 isolates" that are created in the same NodeJS process ("V8 isolate" = JS Heap instance, based on Node GC documentation) :

Among those "V8 isolates", with a proper heap for each, 2 of them have particular high heap amount at the same time :

The main one (Instance : 0000017EBED1F120) goes up to 3.6 GB of heap
A another one (Instance : 0000017FCBC9E030), created at middle-end of build phase, goes up to 2GB

What also have to be noticed is that the second one consumes such a memory ONLY when option "production > optimization > scripts" in "angular.json" is set to true. The "production > optimization > scripts" option in "angular.json" is required for my production performance requirements (minification, tree-shaking, dead-code elimination).

So my questions/remarks are :

The "max_old_space_size" parameter seems to mainly apply to the MAIN V8 isolate, so I'm not able to configure "global" memory consumption during build phase. Is there a parameter to configure the GLOBAL heap used by NodeJS process for building phase ? If not, I think it could be useful to have one, and/or be able to fine-tune memory for all V8 isolates with high memory consumption
I cannot find any documentation explaning the fact that "production > optimization > scripts" option can result in high memory consumption, why it takes such memory, how I could tune it. Is is possible to have such a documentation / explanation ?
I also found this issue about memory limits for several "V8 isolates" in NodeJS, and was wondering whether it could be linked to my issue : worker: resourceLimits overridden by --max-old-space-size

Thanks

Minimal Reproduction

Command run :

node --trace-gc node_modules/@angular/cli/bin/ng build --configuration production

Packages :

  "dependencies": {
    "@angular/animations": "~14.2.0",
    "@angular/cdk": "~14.2.0",
    "@angular/cdk-experimental": "~14.2.0",
    "@angular/common": "~14.2.0",
    "@angular/compiler": "~14.2.0",
    "@angular/core": "~14.2.0",
    "@angular/flex-layout": "^14.0.0-beta.40",
    "@angular/forms": "~14.2.0",
    "@angular/material": "~14.2.0",
    "@angular/material-moment-adapter": "~14.2.0",
    "@angular/platform-browser": "~14.2.0",
    "@angular/platform-browser-dynamic": "~14.2.0",
    "@angular/router": "~14.2.0",
    "@danmarshall/deckgl-typings": "4.9.12",
    "@dashjoin/json-schema-form": "^0.9.0",
    "@deck.gl/aggregation-layers": "8.7.7",
    "@deck.gl/core": "8.7.7",
    "@deck.gl/extensions": "8.7.7",
    "@deck.gl/geo-layers": "8.7.7",
    "@deck.gl/json": "8.7.7",
    "@deck.gl/layers": "8.7.7",
    "@deck.gl/mapbox": "8.7.7",
    "@deck.gl/mesh-layers": "8.7.7",
    "@loaders.gl/core": "3.1.8",
    "@loaders.gl/draco": "3.1.8",
    "@loaders.gl/gltf": "3.1.8",
    "@loaders.gl/worker-utils": "3.1.8",
    "@luma.gl/constants": "8.5.10",
    "@luma.gl/core": "8.5.10",
    "@mapbox/mapbox-gl-draw": "^1.2.0",
    "@nebula.gl/layers": "^1.0.3-rev-2",
    "@nebula.gl/overlays": "1.0.4",
    "@ngx-translate/core": "^14.0.0",
    "@ngx-translate/http-loader": "^7.0.0",
    "@turf/along": "6.3.0",
    "@turf/destination": "6.3.0",
    "@turf/distance": "6.3.0",
    "@turf/helpers": "6.3.0",
    "@turf/length": "6.3.0",
    "angular-oauth2-oidc": "^13.0.1",
    "angular-oauth2-oidc-jwks": "^13.0.1",
    "angular-resize-event": "^3.2.0",
    "angular-tag-cloud-module": "^13.0.0",
    "aws-sdk": "^2.1104.0",
    "bpmn-js": "^11.5.0",
    "chroma-js": "2.1.0",
    "core-js": "^3.21.1",
    "css-loader": "^6.7.1",
    "csscolorparser": "1.0.3",
    "diagram-js": "^11.11.0",
    "echarts": "^5.4.0",
    "file-saver": "^2.0.2",
    "golden-layout": "^1.5.9",
    "html2canvas": "1.3.2",
    "ip-address": "5.9.0",
    "jquery": "^3.4.1",
    "jsep": "0.3.4",
    "jsonata": "^1.8.5",
    "jsoneditor": "^9.1.4",
    "jspdf": "^2.1.1",
    "jspdf-autotable": "^3.5.14",
    "jszip": "^3.10.1",
    "lodash": "^4.17.21",
    "mapbox-gl": "1.13.2",
    "moment": "^2.29.1",
    "mqtt": "4.3.7",
    "ng5-slider": "^1.2.4",
    "ngx-color": "^5.1.4",
    "ngx-echarts": "^14.0.0",
    "ngx-material-timepicker": "^5.5.3",
    "ngx-mqtt": "9.0.5",
    "ngx-papaparse": "^4.0.4",
    "ngx-quill-upload": "^2.0.0",
    "ngx-scrollbar": "^7.4.1",
    "ngx-ui-loader": "^13.0.0",
    "object-hash": "3.0.0",
    "primeicons": "^6.0.0",
    "primeng": "^14.2.1",
    "quill": "^2.0.0-dev.4-rev-1",
    "quill-better-table": "^1.2.10-rev-1",
    "quill-blot-formatter": "v1.0.5-rev-1",
    "quill-delta": "^5.0.0-rev-1",
    "roboto-fontface": "^0.10.0",
    "rxjs": "~7.5.0",
    "rxjs-compat": "^6.6.7",
    "stream": "0.0.2",
    "supercluster": "7.1.3",
    "three": "^0.131.3",
    "timers": "^0.1.1",
    "tslib": "^2.3.0",
    "util": "^0.12.4",
    "uuid": "8.3.2",
    "vis-timeline": "^7.7.0",
    "xml2js": "^0.4.23",
    "zone.js": "~0.11.4"

Exception or Error

No response

Your Environment

_                      _                 ____ _     ___
    / \   _ __   __ _ _   _| | __ _ _ __     / ___| |   |_ _|
   / △ \ | '_ \ / _` | | | | |/ _` | '__|   | |   | |    | |
  / ___ \| | | | (_| | |_| | | (_| | |      | |___| |___ | |
 /_/   \_\_| |_|\__, |\__,_|_|\__,_|_|       \____|_____|___|
                |___/

Angular CLI: 14.2.11
Node: 16.19.1
Package Manager: npm 8.19.3
OS: win32 x64

Angular: 14.2.12
... animations, common, compiler, compiler-cli, core, forms
... language-service, platform-browser, platform-browser-dynamic
... router

Package                            Version
------------------------------------------------------------
@angular-devkit/architect          0.1402.11
@angular-devkit/build-angular      14.2.11
@angular-devkit/core               14.2.11
@angular-devkit/schematics         14.2.11
@angular/cdk                       14.2.7
@angular/cdk-experimental          14.2.7
@angular/cli                       14.2.11
@angular/flex-layout               14.0.0-beta.41
@angular/material                  14.2.7
@angular/material-moment-adapter   14.2.7
@schematics/angular                14.2.11
rxjs                               7.5.7
typescript                         4.6.4

Anything else relevant?

Extract of "angular.json" used for production build :

"production": {
              "fileReplacements": [
                {
                  "replace": "src/environments/environment.ts",
                  "with": "src/environments/environment.prod.ts"
                }
              ],
              "optimization": {
                "scripts": true,
                "styles": true,
                "fonts": true
              },
              "outputHashing": "bundles",
              "sourceMap": false,
              "namedChunks": false,
              "aot": true,
              "extractLicenses": true,
              "vendorChunk": false,
              "buildOptimizer": false,
              "budgets": [
                {
                  "type": "initial",
                  "maximumWarning": "8mb",
                  "maximumError": "30mb"
                }
              ]
            }

Extract of build logs, with GC events (middle-end of build phase, with 2 V8 isolates having high amount of heap at the same time) :

[864:0000017FCBC9E030]    76944 ms: Scavenge 1794.3 (2071.8) -> 1790.7 (2071.8) MB, 2.2 / 0.0 ms  (average mu = 0.983, current mu = 0.982) allocation failure 
[864:0000017FCBC9E030]    76950 ms: Scavenge 1794.4 (2071.8) -> 1790.8 (2071.8) MB, 2.2 / 0.0 ms  (average mu = 0.983, current mu = 0.982) allocation failure 
[864:0000017FCBC9E030]    76958 ms: Scavenge 1794.6 (2071.8) -> 1791.0 (2071.8) MB, 2.5 / 0.0 ms  (average mu = 0.983, current mu = 0.982) allocation failure 
[864:0000017FCBC9E030]    76966 ms: Scavenge 1794.8 (2071.8) -> 1791.1 (2071.8) MB, 2.4 / 0.0 ms  (average mu = 0.983, current mu = 0.982) allocation failure 
[864:0000017FCBC9E030]    76972 ms: Scavenge 1794.9 (2071.8) -> 1791.3 (2071.8) MB, 2.3 / 0.0 ms  (average mu = 0.983, current mu = 0.982) allocation failure 
[864:0000017FCBC9E030]    76980 ms: Scavenge 1795.2 (2072.5) -> 1791.8 (2071.8) MB, 2.5 / 0.0 ms  (average mu = 0.983, current mu = 0.982) allocation failure 
[864:0000017EBED1F120]   380146 ms: Scavenge 3005.1 (3664.5) -> 3004.9 (3660.5) MB, 3.9 / 0.0 ms  (average mu = 0.932, current mu = 0.883) allocation failure 
[864:0000017EC91A0350]    77134 ms: Scavenge 308.7 (524.0) -> 305.4 (520.0) MB, 0.8 / 0.0 ms  (average mu = 0.988, current mu = 0.997) task 
[864:0000017FCCAC1060]    77142 ms: Mark-sweep 37.7 (46.9) -> 11.4 (36.8) MB, 4.0 / 0.0 ms  (+ 0.2 ms in 4 steps since start of marking, biggest step 0.2 ms, walltime since start of marking 9 ms) (average mu = 1.000, current mu = 1.000) finalize incremental marking via task GC in old space requested
[864:0000017EBED1F120]   381101 ms: Mark-sweep 3041.4 (3697.1) -> 2541.2 (3408.1) MB, 932.7 / 0.0 ms  (average mu = 0.984, current mu = 0.989) allocation failure scavenge might not succeed
[864:0000017EBED1F120]   381447 ms: Scavenge 2544.0 (3410.0) -> 2543.5 (3410.0) MB, 12.5 / 0.0 ms  (average mu = 0.984, current mu = 0.989) external memory pressure 
[864:0000017EBED1F120]   381451 ms: Scavenge 2543.5 (3410.0) -> 2543.6 (3410.0) MB, 3.5 / 0.0 ms  (average mu = 0.984, current mu = 0.989) external memory pressure 
[864:0000017EBED1F120]   382769 ms: Mark-sweep (reduce) 2543.7 (3410.0) -> 2538.6 (3096.7) MB, 72.6 / 0.1 ms  (+ 903.8 ms in 923 steps since start of marking, biggest step 2.8 ms, walltime since start of marking 999 ms) (average mu = 0.963, current mu = 0.415) finalize incremental marking via task GC in old space requested
[864:0000017EBED1F120]   395967 ms: Mark-sweep 2538.9 (3096.7) -> 2538.6 (3069.2) MB, 41.2 / 0.1 ms  (+ 789.9 ms in 1294 steps since start of marking, biggest step 19.7 ms, walltime since start of marking 850 ms) (average mu = 0.954, current mu = 0.937) finalize incremental marking via task GC in old space requested
[864:0000017EBED1F120]   414545 ms: Scavenge 2542.9 (3069.4) -> 2540.1 (3069.2) MB, 15.3 / 0.0 ms  (average mu = 0.954, current mu = 0.937) allocation failure 
[864:0000017EBED1F120]   414559 ms: Scavenge 2542.7 (3069.2) -> 2539.9 (3069.2) MB, 3.9 / 0.0 ms  (average mu = 0.954, current mu = 0.937) allocation failure 
[864:0000017EBED1F120]   414580 ms: Scavenge 2543.6 (3069.2) -> 2540.4 (3069.2) MB, 3.8 / 0.0 ms  (average mu = 0.954, current mu = 0.937) allocation failure

clydin commented 11 months ago

Angular v14 is no longer supported. Please consider updating to a more recent and supported version. There have been significant improvements in build performance since the v14 timeframe. For more information regarding currently supported Angular versions, please see: https://angular.dev/reference/releases#actively-supported-versions

If you are still encountering issues after updating, please open an issue with updated project information.

angular-automatic-lock-bot[bot] commented 10 months ago

This issue has been automatically locked due to inactivity. Please file a new issue if you are encountering a similar or related problem.

Read more about our automatic conversation locking policy.

_{This action has been performed automatically by a bot.}

angular / angular-cli