Buildtime increased by 50% in v12

[kemsky]

🐞 Bug report

Command (mark with an x)

• [x] build

Description

This is v11.2.13:

$ ng build --prod
√ Browser application bundle generation complete.
√ Copying assets complete.
√ Index html generation complete.

Initial Chunk Files                   | Names         |      Size
main.1bbe17e891e0e81a6bae.js          | main          |   3.20 MB
scripts.5b3322e07a3cdcf6af69.js       | scripts       |   1.48 MB
styles.19406af932cc90ae2667.css       | styles        | 152.44 kB
polyfills-es5.2cb7395a5fd592fe7443.js | polyfills-es5 |  85.88 kB
runtime.23c71f07ba7ec5c3766a.js       | runtime       |   1.51 kB

                                      | Initial Total |   4.92 MB

Build at: 2021-05-14T11:00:10.476Z - Hash: 034e3176df64cbc0b60a - Time: 88357ms

Verbose log (logging itself causes slowdown, unfortunately...): angular11.txt

Angular CLI: 11.2.13
Node: 14.17.0
OS: win32 x64

Angular: 11.2.13
... animations, cli, common, compiler, compiler-cli, core, forms
... language-service, platform-browser, platform-browser-dynamic
... router
Ivy Workspace: Yes

Package                         Version
---------------------------------------------------------
@angular-devkit/architect       0.1102.13
@angular-devkit/build-angular   0.1102.13
@angular-devkit/core            11.2.13
@angular-devkit/schematics      11.2.13
@angular/cdk                    11.2.12
@angular/material               11.2.12
@schematics/angular             11.2.13
@schematics/update              0.1102.13
rxjs                            6.6.7
typescript                      4.1.5

This is v12 with yarn.lock and node_modules removed and reinstalled:

$ ng build --configuration production
Warning: Support was requested for IE 11 in the project's browserslist configuration. IE 11 support is deprecated since Angular v12.
For more information, see https://angular.io/guide/browser-support
√ Browser application bundle generation complete.
√ Copying assets complete.
√ Index html generation complete.

Initial Chunk Files                   | Names         |      Size
main.25755cb56d908f1ffa0c.js          | main          |   3.19 MB
scripts.5b3322e07a3cdcf6af69.js       | scripts       |   1.48 MB
styles.51baf6c2a9fbe6837aa1.css       | styles        | 154.28 kB
polyfills-es5.b787b17c5e8d81a4b18e.js | polyfills-es5 |  90.75 kB
runtime.bf3db8034d6e10a74a56.js       | runtime       |   1.20 kB

                                      | Initial Total |   4.92 MB

Build at: 2021-05-14T11:18:16.175Z - Hash: 8806f0a3a3b1271495e0 - Time: 157125ms

Verbose log (logging itself causes slowdown, unfortunately...): angular12.txt

Angular CLI: 12.0.0
Node: 14.17.0
Package Manager: yarn 1.22.5
OS: win32 x64

Angular: 12.0.0
... animations, cdk, cli, common, compiler, compiler-cli, core
... forms, language-service, material, platform-browser
... platform-browser-dynamic, router

Package                         Version
---------------------------------------------------------
@angular-devkit/architect       0.1200.0
@angular-devkit/build-angular   12.0.0
@angular-devkit/core            12.0.0
@angular-devkit/schematics      12.0.0
@schematics/angular             12.0.0
rxjs                            6.6.7
typescript                      4.2.4

We have a single small *.scss file (overriding some material colors) and it was updated by migration. The application has only one module. I tried removing scss and css from angular.json, but it is still slower than v11.

On average it is 50% slower.

-------------------------------------------------------------------------------
Language                     files          blank        comment           code
-------------------------------------------------------------------------------
TypeScript                    1011          10391           2652          54543
HTML                           311            629              7           7832
CSS                             66            344             30           2109
Sass                             1             15             14             63
JavaScript                       1              9              3             55
JSON                             1              0              0              3
-------------------------------------------------------------------------------
SUM:                          1391          11388           2706          64605
-------------------------------------------------------------------------------

---

Recorded profile using node 12:

https://link.nithins.me/s2Uca0Ti

[alan-agius4]

@the-ult, from the profiler at appears that it might be caused by NX. Can you replicate using only the Angular CLI?

[the-ult]

Don't know how atm. Since it automatically uses the nx build command. 🤔

Might be related to https://github.com/nrwl/nx/issues/5370

[alan-agius4]

Yesterday release (12.0.1) contains several performance improvements. Please give it a try!

[the-ult]

Build is indeed faster (more like V11) 👍 Now: 49.728mswas 156.175ms with v12.0.0

[kemsky]

I can not confirm that it is faster, it is the same (NG_BUILD_CACHE=0, no scss files):

yarn run v1.22.5
$ ng build --configuration production
Warning: Support was requested for IE 11 in the project's browserslist configuration. IE 11 support is deprecated since Angular v12.
For more information, see https://angular.io/guide/browser-support
√ Browser application bundle generation complete.
√ Copying assets complete.
√ Index html generation complete.

Initial Chunk Files                   | Names         |     Size
main.4005861e09f6e41ffcfa.js          | main          |  3.25 MB
scripts.a1b5cf0dcd279cdb7b01.js       | scripts       |  1.48 MB
polyfills-es5.4c8a7ba5c0469b219d89.js | polyfills-es5 | 90.75 kB
runtime.123c72c8729cb6d2b266.js       | runtime       |  1.20 kB

                                      | Initial Total |  4.82 MB

Build at: 2021-05-20T20:02:02.300Z - Hash: 6163206ae2d71ff8308a - Time: 161921ms

Also, I noticed a huge CPU load at the beginning of the build process:

Angular CLI: 12.0.1
Node: 14.17.0
Package Manager: yarn 1.22.5
OS: win32 x64

Angular: 12.0.1
... animations, cdk, cli, common, compiler, compiler-cli, core
... forms, language-service, material, platform-browser
... platform-browser-dynamic, router

Package                         Version
---------------------------------------------------------
@angular-devkit/architect       0.1200.1
@angular-devkit/build-angular   12.0.1
@angular-devkit/core            12.0.1
@angular-devkit/schematics      12.0.1
@schematics/angular             12.0.1
rxjs                            6.6.7
typescript                      4.2.4

Done in 2.19s.

There is a way to improve performance noticeably: cross-env NODE_OPTIONS=--max_old_space_size=8048 ng build --configuration production builtimes are now 10s less.

[cypressious]

I've just upgraded a project to 12.0.1 and I'm seeing significant increases in serve/build times. I've recorded profiles of both v11 and v12 and I'm seeing a suspicious gap in v12 where the process is mostly idle .

Angular 12 profile: https://www.file.io/download/tr8wrukOKzKh

Here's I've zoomed in on a bit in the mostly empty gap section:

Angular 11 profile for comparison: https://www.file.io/download/3t39VkJLfeat

[alan-agius4]

@cypressious, can you please re-upload the files? I am trying to look into this but I am getting the following error file.io The file you requested has been deleted

[cypressious]

@alan-agius4 please try https://filehorst.de/d/dvgbldhr. Sorry for the crappy file hoster, don't know where else to upload the file.

Alternative link that expires in 7 days: https://www.dropbox.com/t/35SR07Q2zYOTWIYa

[danielehrhardt]

I've just upgraded a project to 12.0.1 and I'm seeing significant increases in serve/build times. I've recorded profiles of both v11 and v12 and I'm seeing a suspicious gap in v12 where the process is mostly idle .

Angular 12 profile: https://www.file.io/download/tr8wrukOKzKh

Here's I've zoomed in on a bit in the mostly empty gap section:

Angular 11 profile for comparison: https://www.file.io/download/3t39VkJLfeat

Still really slow with 12.0.1 and a breaking Issue with zIndex CSS Parsing

[alan-agius4]

@alan-agius4 please try https://filehorst.de/d/dvgbldhr. Sorry for the crappy file hoster, don't know where else to upload the file.

Alternative link that expires in 7 days: https://www.dropbox.com/t/35SR07Q2zYOTWIYa

Is that the full profile? I am seeing a different from the one shown above. Pretty much I am not seeing anything in the first 4 seconds.

[cypressious]

@alan-agius4 Yet, that's the one. This is how it looks when I load it in Chrome's Node.js DevTools:

Which tool are you using?

[alan-agius4]

Chrome’s profiler

[cypressious]

I'm doing the following:

• Open chrome://inspect
• Click the Open dedicated DevTools for Node link
• Click Load and select the file

[alan-agius4]

@cypressious, I re-downloaded the file from Dropbox and now I can see the same thing. Will investigate this more tomorrow.

[clydin]

That gap is the time taken by terser (the JS optimizer). Webpack runs terser in workers which causes the main thread to remain idle while the optimizer works since there is nothing left for the build process to do at that point (barring the final write to disk).

[cypressious]

Why do we need optimization in dev mode?

[JoostK]

Why do we need optimization in dev mode?

So I have been wondering whether the switch to prod builds by default could have affected development builds. I believe that the CLI performs a migration of angular.json to switch the builder options to be targeting development mode as before, but maybe there are cases where the project ends up being misconfigured.

[clydin]

A production (or development) build is defined purely by the options present in the configuration section. If the above build was not intended to be a production build then the project configuration may be incorrect.

[alan-agius4]

Are you using 3rd party builders by any chance?

The production by default migration is actually an opt-in migration.

What is happening here is probably one of below things.

1. ng-update was run multiple times, one of the migrations is non-idempotent. Which will cause the project to be miss configured if run multiple times.
2. The migration was not run because you are using unofficial builders which cause un-desired behaviour. In which case, we cannot know how these builders interact with the builder options and therefore we cannot modify the configuration. As such it is recommend that such builders provide their own ng-update migration or at least document what needs to change for version 12.

The migration I am referring too is https://github.com/angular/angular-cli/blob/master/packages/schematics/angular/migrations/update-12/update-angular-config.ts

[cypressious]

Thank you for ideas.

I didn't run ng-update on the project in question. I was under the impression that upgrading the dependencies in the package.json was enough. My bad. I'll try reverting and running ng-update and will report back if it helped.

[clydin]

The https://update.angular.io site can provide update instructions and guidance when updating between Angular versions.

[intellix]

When doing the ng update, I also noticed that it added production by default but it never added these to build/serve and I had to add them myself:

"defaultConfiguration": "development"

[clydin]

production by default is an optional migration. The migration that is run automatically adjusts the options to maintain the previous development by default behavior because several of the options had default values swapped.

[JoostK]

As for the source mapping performance regression: I tracked it down to a change in Webpack 5 and opened webpack/webpack-sources#109, which implements a proposal to resolve the regression. I'm not sure if the changes in that PR are entirely desirable so we'll see where it gets us.

[zwjohn]

Can anyone show me how to use NG_BUILD_CACHE=0, where to add this parameter? I am also getting crashes after upgrading to 12.0.1, build time is much longer and getting crashes due to memory issue.

[cypressious]

I can confirm that following the upgrade instructions migrated the angular.json and fixed the problem.

For those interested, it added

"aot": false,
"vendorChunk": true,
"extractLicenses": false,
"buildOptimizer": false,
"sourceMap": true,
"optimization": false,
"namedChunks": true

to projects.app.architect.build.options.

[zwjohn]

I can confirm that following the upgrade instructions migrated the angular.json and fixed the problem.

For those interested, it added

"aot": false,
"vendorChunk": true,
"extractLicenses": false,
"buildOptimizer": false,
"sourceMap": true,
"optimization": false,
"namedChunks": true

to projects.app.architect.build.options.

I had the same config except aot:false , aot is defaulted to true, and it worked no problem in V11, why should we turn it off in V12?

[clydin]

The migration adjusts the options to provide equivalent behavior from prior to the update. The new/adjusted set of options and their values may be different for each project depending on the starting options. If AOT was enabled before, it will continue to be enabled afterwards but without the option present since the default is now true.

[the-ult]

The recommended way is "aot": true (or not set) => since it is de default. Right?

[intellix]

When I migrated my AOT was true before and set to false. I had to manually set it back to true. But guess I'd need to create a separate repository/reproduction to prove it

[NateRadebaugh]

Running that "optional" migration didn't improve anything for my app in CI but changing to use an Ubuntu build agent vs the windows build agent did improve time a bit, but the builds are still considerably slower for my azure devops pipelines on 12.0.1 than 11.x

[clydin]

The migration run during the recommended update process is not the optional migration. There is also an optional production-by-default migration available in v12 that changes the configuration to cause ng build (with no options) to build for production and ng serve (with no options) to build for development. This optional migration can be executed via: ng update @angular/cli --migrate-only production-by-default

[zwjohn]

I tried to adjust all possible parameters and didn't help much on the memory usage for V12. I had to revert back to V11 for my project. Hoping there will be a solution for the next release

[the-ult]

With the "@angular-devkit/build-angular": "12.0.1", (and 12.0.2) somehow the build fails on gitlab when using NX

@see https://github.com/nrwl/nx/issues/5729

[simeyla]

If AOT was enabled before, it will continue to be enabled afterwards but without the option present since the default is now true.

@clydin This is true the first time you run the migration, but if you then run it again it will actually get set to false (because it see that the value is effectively false but not explicitly false so it then makes it explicitly false). I'm going to report this as a separate issue.

[clydin]

Yes. That migration is not idempotent. However, that would only apply if the project was previously using prerelease versions of v12 or the automatic migration was specifically rerun with the “—migrate-only” option. The former requires special care regardless due to migration changes (potentially breaking) prior to a stable release; and the later is only recommended if “ng update” was not used to update the package versions.

[jcompagner]

v12.0.3 does have a slight improvement now (i guess because of the fix in webpack about the sourcemaps?) but the improvement is like 5%, for the testcase in #20989 (v12: 100s -> 95s, v11: 50s)

The double compile in watch mode is gone but still the watch mode itself, when changing a file, is much slower

it seems that the "sealing" phase is the one that really takes a lot of time for me.

[alan-agius4]

@jcompagner,

I looked at your repo and I don't see that much of gap between v11 and v12, in rebuilds I did see a big difference, but not as described. What I noticed was that re-builds were faster in v12.

• First build without any disk cache rm -rf node_modules/.cache took ~121s in v11 and ~134s v12.
• Second cold build took ~53s in v11, ~58s in v12.
• Incremental re-build in v11 took 16.5s, in v12 the same change takes ~7.7s. (Note: my change was something that effects the output such as changing a string value)

[jcompagner]

right you are correct with incremental build when you actually are inserting code..

but for the rest, i did do a retest: (now with 12.0.3)

" V12 12.zip : i start with

npm install

npm run build (with ngcc compile): 158s

npm run build (second time): 94s

npm run build (third time): 94s

npm run build_debug: 111s

adding "console.log("test");" to src\sablo\sablo.service.ts constructor: 10s

removing "console.log("test");" to src\sablo\sablo.service.ts constructor: 7.4s

npm run build_debug (second time): 46s

adding "console.log("test");" to src\sablo\sablo.service.ts constructor: 9.6s

removing "console.log("test");" to src\sablo\sablo.service.ts constructor: 7.0s

V11: 11.zip i start with

npm install

npm run build (with ngcc compile): 159s

npm run build (second time): 46s

npm run build (third time): 47s

npm run build_debug: 106s

adding "console.log("test");" to src\sablo\sablo.service.ts constructor: 17s (for some reason it also generated styles again)

removing "console.log("test");" to src\sablo\sablo.service.ts constructor: 14s

npm run build_debug (second time): 45s

adding "console.log("test");" to src\sablo\sablo.service.ts constructor: 18s (for some reason it also generated styles again)

removing "console.log("test");" to src\sablo\sablo.service.ts constructor: 16s "

so yes adding a console.log() statement is slower in v11, and an empty line is for some reason a lot faster..

But still the actual production build is way slower for me. (after ngcc because the it seems to be the same)

i see you are on linux, i am on windows. could that be the big difference? i will ask a co worker to have the same test for me also on linux

[clydin]

Can you run those production cases with the environment variable NG_BUILD_CACHE set to 0?

[jcompagner]

that doesn't matter for me tested this now also on Ubuntu on WSL2 under windows 10 but the numbers also there are the same (with our without that NG_BUILD_CACHE)

[clydin]

v11 caches the result of the JavaScript optimizer (terser) but v12 (with Webpack 5) does not. There should definitely be a difference with the v11 numbers.

[jcompagner]

ah you want to slow down 11 yes by using that 11 is as slow as 12

same under linux:

11: Build at: 2021-06-03T13:08:11.899Z - Hash: 70cb17ecb1f07c22904b - Time: 197106ms Build at: 2021-06-03T13:09:50.550Z - Hash: 70cb17ecb1f07c22904b - Time: 55809ms Build at: 2021-06-03T13:12:58.205Z - Hash: 70cb17ecb1f07c22904b - Time: 53705ms

12: Build at: 2021-06-03T13:16:39.392Z - Hash: 3891f04b7906cecddb79 - Time: 195291ms Build at: 2021-06-03T13:18:59.888Z - Hash: 3891f04b7906cecddb79 - Time: 119048ms Build at: 2021-06-03T13:21:28.220Z - Hash: 3891f04b7906cecddb79 - Time: 119192ms

with NG_BUILD_CACE=0

11: Build at: 2021-06-03T13:31:57.442Z - Hash: 70cb17ecb1f07c22904b - Time: 121182ms 12: Build at: 2021-06-03T13:26:28.629Z - Hash: 3891f04b7906cecddb79 - Time: 124241ms

[danielehrhardt]

If you still have Problems, try to run: ng update @angular/cli --migrate-only --from 11 --to 12

[blackfan23]

I ended upp generating a new v12 app and migrated all the app code manually. Took a while but now build-times are better than ever.

[jcompagner]

If you still have Problems, try to run: ng update @angular/cli --migrate-only --from 11 --to 12

i don't get this, should i do that? on what an already converted to 12? if i do that then aot:false is added and nothing more which is just bogus (i thought this was fixed but i am using 12.03 and i still get that)

[simeyla]

For anyone struggling with ng serve times under Angular 12, the following may help - especially if you're only working on a subset of your application at a time.

Create a new build configuration architect/build/configurations/quick in angular.json

     "quick": {

          "aot": false,

          "sourceMap": false,

          "fileReplacements": [
            {
              "replace": "src/app/app-routing.module.ts",
              "with": "src/app/app-routing.module.quick.ts"
            }
          ]
        }

Under architect/serve/configurations add:

        "quick": {
            "browserTarget": "DEFENDER:build:development,quick"
        }

Notice that this targets both development and quick.

Then pick and choose from the options shown above:

• aot - disable for a faster build. I strongly recommend testing with a normal development build regularly to avoid surprises.
• sourceMap - disable sourceMap for much faster build

Then the biggest saving:

• fileReplacements - this replaces your app.routing with a cut down copy containing only the routes you're working on. It's a copy so you don't accidentally deploy it! Copy your main file, comment out the routes and comment out obsolete includes. This halved the time of my build and is very useful when working only on a small subset of my site. *

Run it with:

ng serve -c quick

I like explicitly calling it a quick build so as not mess with development settings.

And before anyone says 'but it's just the first build that is slow' - I know, but I have to do first builds frequently enough. Also I'm aware of nx which I tried, but it doesn't seem to help my case which is a single large project.

Tip: It is quite likely that you will see a lot of compiler warnings if you comment out many routes in your file. This is to be expected, and is actually desirable. What it means is your dependency tree isn't optimal. It took me a few hours to fix mine. The primary problem was I'd often define an interface in the same file as a component and then reference that interface from another component in a different module (route). When typescript pulls in the interface for the second module it compiles the whole component file, which is not only unnecessary but results in errors since the component isn't imported in a module that is known. Took some time to fix, but I'm much happier with my file structure and build time after doing this.

Still hoping for webpack 5 and typescript improvements!!

[danielehrhardt]

If you still have Problems, try to run: ng update @angular/cli --migrate-only --from 11 --to 12

If the Migration does not work: angular.json -> projects -> app -> architect -> build-> options

            "aot": true,
            "vendorChunk": true,
            "extractLicenses": false,
            "buildOptimizer": false,
            "sourceMap": true,
            "optimization": false,
            "namedChunks": true

[JoostK]

Please note that the migration would not have set "aot: false"; I'd recommend keeping that turned on if at all possible.

[zwjohn]

Turning aot to false is not a good solution, that simply removes an important feature of CLI. If you had aot turned on with v11 and should keep it on for v12. The performance issue with ng serve is a separate issue.