custom webpack config with compression-webpack-plugin only compress es2015

[steve3d]

Describe the Bug

I believe there are so many people want to use compression-webpack-plugin to compress the generated js file and use it with static_gzip of nginx, and I'm one of these people.

I've used this great package for years, today I suddenly found that with angular 8.2, compression-webpack-plugin only compress the es2015 version of js files, the es-5 version is left uncompressed.

Minimal Reproduction

normal config like the readme, and here is my extra-webpack.config.js

const webpack = require('webpack');
const CompressionPlugin = require('compression-webpack-plugin');

module.exports = (config, options) => {

  // compress everything for nginx static_gzip
  config.plugins.push(
    new CompressionPlugin({
      test: /\.(js|css|svg|woff|woff2)$/,
      compressionOptions: { threshold: 8192, level: 9}
    })
  );

  return config;
};

with this simple config, the compress-plugin will only compress es2015 version.

Expected Behavior

should able to compress both es2015 and es-5 version of js.

Screenshots

Environment

Libs
- @angular/core version: 8.2.14
- @angular-devkit/build-angular version: 0.803.19
- @angular-builders/custom-webpack version: 8.3.0

For Tooling issues:
- Node version: v10.17.0
- Platform:  only tested on Mac/Linux

[steve-todorov]

The issue here is related to how angular-cli is actually bootstrapping webpack. For example, the index.html file is actually generated outside webpack and after it has successfully exited. In my case I was trying to create a zip out of the dist/app folder, but the index.html file was missing there. After a ton of wasted time in digging / debugging I noticed that angular_devkit/build_angular/src/browser/index.ts is actually doing writeIndexHtml (line 513) only after webpack has completed.

I assume this issue is related to that same part of the code since the differential bundles are loaded after webpack has completed (search for Generating ES5 bundles for differential loading... in the mentioned ts)

[just-jeb]

You're right, index.html is not generated as part of Webpack build and you have indexTransform option to work it around.
However, all the bundles are generated with Webpack and from what I can tell the custom webpack transformation is applied to all of them.
Which means the plugin should be applied to es2015 as well as to es5.
I'd suggest you to put debug print there and see if you're getting your plugin in both configurations. If yes then the problem is not within the builder but rather somewhere in Angular CLI. Maybe the post-processing they perform on the resulting bundle.
If one of the configs lacks CompressionPlugin then it might be a problem with the builder that should be investigated.

[steve3d]

I'd suggest you to put debug print there

how should I do that?

[just-jeb]

Go to dist folder of angular_devkit in node_modules, locate the compiled .js file (browser/index.js) and add a print.

[steve-todorov]

In our case the indexTransform workaround doesn't actually fix anything since we don't actually need to transform the index.html. We actually wanted to:

1. compress all assets so that we have .gz and .br versions for browsers which support it.
2. create an asset-manifest.json
3. finally a zip of dist/app/** which is used down the line in our build pipeline.

I was hoping we could use the custom-webpack builder to achieve all of this via webpack, but since angular-cli has steps happening outside of webpack we cannot reliably use custom-webpack in our build flow.. :/

Our current solution is to have an entirely separate webpack.config which goes over dist/app/** and does the mentioned above steps in a postbuild script we added into the package.json. So when you npm run build it would also do the rest.

Unfortunately I don't have time to debug further into angular_devkit/build_angular right now. I can create a simple hello-world repository which reproduces the compression issue if that would be of any help to you?

[just-jeb]

That would be wonderful, thank you!

[steve-todorov]

Sure, here's the repo. This is just the hello world from the latest Angular 8 with some minor tweaks. When you run npm run build it should produce the mentioned issue. I've attached a screenshot and If you pay close attention to the *-es5-* assets you'll see they have not been compressed. For example main-es5.e14fa0110cab463626ac.js. You can also check the assets-manifest.json which should have the es5 assets included, but does not. This is why I thought the es5 differential assets are actually generated after the entire build - maybe something like the index.html file.

[arturovt]

Seems like it's an Angular's issue. Compression plugin "subscribes" on the emit hook and then runs compressions on files:

compiler.hooks.emit.tapAsync({
    name: 'CompressionPlugin'
}, (compilation, callback) => {
    console.log(Object.keys(compilation.assets));
})

All files are present except of main-es5 :thinking:

I guess it's related to that differential loading process, emit hook is tapped right before assets are emitted to the dist folder, but the generation of ES5 bundles is started after that, you can open _dist/projectname/static/assets and see that es-2015.js.gz files are emitted before es-5.js are generated.

I'm not a fan of that solution but you could try it as a temporary workaround. You could run gzip right after build command, like:

npm run build && gzip -rk9 dist

Same with brotli after installing brotli CLI.

[just-jeb]

@steve-todorov Please take a look at this comment. es5 bundles are not generated as part of the Webpack build, therefore any plugin you specify in your custom webpack config won't be applied to them.

[SeanLatimer]

Interesting, I have a project I was setting up but the only file not compressed is main-es5.xxx.js.

Something weird must be happening because when I had a console log in the compression plugin it showed all the es5 bundles other than main

[just-jeb]

@SeanLatimer Differential loading enabled? You're getting both es2015 and es5 bundles?

[SeanLatimer]

Both enabled, yes. I found it odd that only main-es5.xxx.js would not be compressed.

I assumed it may be a race condition of some sort.

[just-jeb]

It's really weird, because as I said, the es5 bundles are not generated during the Webpack build (thus shouldn't be affected by Webpack plugins). Can you reproduce it in an isolated environment?

[arturovt]

@just-jeb it's reproducible as I mentioned in my comment.

[just-jeb]

@arturovt But es5 files shouldn't be in compilation.assets at all. That's what I can't understand.

[SeanLatimer]

The only es5 assets that seem to be getting into compilation.assets are the runtime and/or polifill bundles. This run didn't get the runtime for some reason.

[arturovt]

@arturovt But es5 files shouldn't be in compilation.assets at all. That's what I can't understand.

I understand your point. But it DOES, there are all files except of main :thinking:

[just-jeb]

The es5 polyfills chunk is a separate entry point that is built when ES5 support is needed. Unlike the other files, it’s not a downleveled version of the es2015 polyfillls chunk

FYI