webpack 5 <-> html-webpack-plugin 5

jantimon commented 4 years ago

During the beta I tried to provide a html-webpack-plugin version which is compatible to webpack 4 and webpack 5.
As there have been some API and typing changes in webpack 5 this approach is limited.

To provide the best possible webpack 5 support I have started on a dedicated webpack 5 version of the html-webpack-plugin.

current progress:

[x] drop webpack 4 support
[x] drop node < 10 like webpack
[x] make use of Compiler.hook.initialize to get the latest options
[x] update typings for webpack 5
[x] fix scriptLoading defer for inject false
[x] make scriptLoading defer the default
[x] add assets in Compilation.PROCESS_ASSETS_STAGE_ADDITIONAL (see https://github.com/webpack/webpack/issues/11425#issuecomment-686606318 )
[x] add assets with new api compilation.emitAsset (#1522)
[x] use new webpack.sources.RawSource(html) to create assets
[x] add support for [name]
[x] provide options and output name in all hooks
[x] allow filename to be a function
[x] emitAssets even if they have been generated in a previous compilation (to fix #1476 and #1432)
[x] use compiler.webpack.LibraryTemplatePlugin
[x] verify support for raw-loader (blocked by https://github.com/webpack/webpack/issues/11909)
[x] remove @types/tapable as it is included in tapable 2
[x] provide a publicPath to the child compiler to add support for publicPath: 'auto' & 'asset/resource' (https://github.com/webpack/webpack/issues/11968#issuecomment-725117719)
[x] use the correct stage to allow sri and service worker generation (blocked by https://github.com/webpack/webpack/issues/11822)
[x] fix error reporting (which is currently breaking the travis build)
[x] use the html-webpack-plugin publicPath option also for the template child compilation
~~remove html-minification (if a separate plugin which covers the html minification is available)~~ (postponed)

current blockers (help needed by the webpack team):

[x] webpack/issues/11822
[x] webpack/issues/11909
[x] https://github.com/webpack/webpack/issues/12214

current alpha playround:

CodeSandbox of the html-webpack-plugin 5.0.0-alpha.17 html-webpack-plugin_5_x_alpha_-_CodeSandbox

Most of the work is done just by me in my spare time for fun - so if this work helps you feel free to buy me a beer 🍺

jeffposnick commented 4 years ago

FWIW, webpack v4.40.0 added the new Asset API, so you may not have to drop webpack v4.x support completely, if you require that users update to at least that version.

(It's still pretty inconvenient to have to hook into the compilation in two fundamentally different ways, depending on whether you're using webpack v4.x or webpack v5.x, though.)

tianyingchun commented 4 years ago

good news, waiting 👍

tianyingchun commented 4 years ago

and When is it expected to be released？

jantimon commented 4 years ago

@jeffposnick I have an idea to fix your issue - can you please try the next branch?

Instead of extracting the result of the child compiler it will now extract the assets inside a child compiler hook:

const extractedAssets = [];
      childCompiler.hooks.compilation.tap('HtmlWebpackPlugin', (compilation) => {
        compilation.hooks.processAssets.tap(
          {
            name: 'HtmlWebpackPlugin',
            stage: Compilation.PROCESS_ASSETS_STAGE_ADDITIONS
          },
          (assets) => {
            temporaryTemplateNames.forEach((temporaryTemplateName) => {
              if (assets[temporaryTemplateName]) {
                extractedAssets.push(assets[temporaryTemplateName]);
                compilation.deleteAsset(temporaryTemplateName);
              }
            });
          }
        );
      });

@tianyingchun hard to say - depends a little bit on my time :)

jeffposnick commented 4 years ago

I'm seeing different behavior, but not having luck getting access to the assets created by html-webpack-plugin from within my plugin yet. I'm using the following devDependencies:

"html-webpack-plugin": "github:jantimon/html-webpack-plugin#next",
"webpack": "webpack@^5.1.0"

First off, I see the following originating from html-webpack-plugin:

(node:79288) [DEP_WEBPACK_COMPILATION_ASSETS] DeprecationWarning: Compilation.assets will be frozen in future, all modifications are deprecated.
BREAKING CHANGE: No more changes should happen to Compilation.assets after sealing the Compilation.
        Do changes to assets earlier, e. g. in Compilation.hooks.processAssets.
        Make sure to select an appropriate stage from Compilation.PROCESS_ASSETS_STAGE_*.

And at the time my plugin runs, I don't see the expected index.html in either the main compilation's assets, or the child compilation's assets. (So this part is different in the next branch, as I was previously seeing the incorrectly named index.html in the child compilation's assets.)

I am curious as to whether you really need to go through the steps of deleting your assets from the child compilation and adding them to the parent compilation—maybe someone from the webpack team could weigh in on the best practice? There is a mode in the plugin that I support in which a child compilation is performed (independent of the logic related to getting a list of all the assets), and I just leave the assets it creates as part of the child compilation, and don't attempt to "move" them to the parent compilation.

My plugin, at least, already iterates over both the parent and any child compilations to get the asset list from each.

tianyingchun commented 4 years ago

💯 My projects totally want to upgrade to webpack@5 exception html-webpack-plugin cause of i have some cusomized html plugin based on the hooks of html-webpack-plugin if you have ready for html-webpack-plugin@next i can install as devDeps to do some testings works. :)

jantimon commented 4 years ago

@jeffposnick okay good to know that the temporary file disappeared - now I just need to use the same api to add it once again.

Deleting the compiled template from the child compilation as it will be processed even further before it is added again with the final name. It also allows to compile a template only once even if it is used to generate multiple html files.

jantimon commented 4 years ago

There seems to be a blocker - if I move the asset additions away from the emit phase the child compilation is not done.

jantimon commented 4 years ago

I pushed the current version as html-webpack-plugin@5.0.0-alpha.1 which you should be able to install with html-webpack-plugin@next

jantimon commented 4 years ago

Wow @sokra improved the deleteAsset function https://github.com/webpack/webpack/pull/11704 so now we can use thisCompilation for the child compiler. 👍

jantimon commented 4 years ago

I have just released html-webpack-plugin@5.0.0-alpha.2 (which requires webpack 5.1.2) - it is now adding the assets the right way @jeffposnick could you please try it once again?

tianyingchun commented 4 years ago

@jantimon i tried webpack@5.1.2 & html-webpack-plugin@5.0.0-alpha.2 it also throw error

(node:48122) [DEP_WEBPACK_COMPILATION_ASSETS] DeprecationWarning: Compilation.assets will be frozen in future, all modifications are deprecated.
BREAKING CHANGE: No more changes should happen to Compilation.assets after sealing the Compilation.
        Do changes to assets earlier, e. g. in Compilation.hooks.processAssets.
        Make sure to select an appropriate stage from Compilation.PROCESS_ASSETS_STAGE_*.

TypeError: The 'compilation' argument must be an instance of Compilation

jantimon commented 4 years ago

Thanks for testing @tianyingchun - it looks like this release didn't include all changes - can you please try one more time with Now it's part of html-webpack-plugin@5.0.0-alpha.3 ?

Thebarda commented 4 years ago

Thanks for testing @tianyingchun - it looks like this release didn't include all changes - can you please try one more time with Now it's part of html-webpack-plugin@5.0.0-alpha.3 ?

I don't have this problem anymore. Well done 👍

tianyingchun commented 4 years ago

@jantimon yes the [DEP_WEBPACK_COMPILATION_ASSETS] DeprecationWarning has disappeared. but have another questions about the hooks of html-webpack-plugin i write a plugin based on the hooks thisCompilation as bellow

apply(compiler: Compiler): void { 
    compiler.hooks.thisCompilation.tap(this.pluginName, (compilation) => {
      compilation.mainTemplate.hooks.requireExtensions.tap(this.pluginName, () => {
....

finnally it give an error

TypeError: The 'compilation' argument must be an instance of Compilation

jantimon commented 4 years ago

@tianyingchun I am not sure how this would be connected to the html-webpack-plugin.. if it is could you please open a new issue for that part?

tianyingchun commented 4 years ago

ok， i will research this issue first, thanks very much. 👍

tianyingchun commented 4 years ago

alpha-5 still can not success build it throws error like https://github.com/jantimon/html-webpack-plugin/issues/1451

serprex commented 4 years ago

alpha.6 is working fine with my simple webconfig. I say simple since 4.5.0 also was working while using webpack 5

akphi commented 4 years ago

@jantimon Hi looks like in v5.alpha.6, favicon is breaking. This is my repo. My case is that I have a favicon.ico file of around 14kb, when I do yarn build I can see it gets emitted to favicon.ico of 27kb, and this icon is clearly corrupted.

This works fine for 5.alpha.6, the favicon emitted is 14kb and works just fine.

jantimon commented 4 years ago

@akphi we convert the favicon into a RawSource - maybe that's wrong:

https://github.com/jantimon/html-webpack-plugin/blob/1696eff5c0c029fecb9bd3f9efbf1c2a81857aa4/index.js#L467

Can you please try to find out if there is a better way to add binary files to the webpack compilation?

akphi commented 4 years ago

@jantimon I just went through the code for RawSource.

From the list they have in webpack-sources, not sure if I could suggest a better one. But, why do we set convertToString=true here?

https://github.com/jantimon/html-webpack-plugin/blob/1696eff5c0c029fecb9bd3f9efbf1c2a81857aa4/index.js#L467

jantimon commented 4 years ago

I couldn't find any API documentation about the best usage and tried true

Can you please try to change it to false locally? Maybe it works better that way for binary files

akphi commented 4 years ago

@jantimon sure, I mean, the doc is sorta outdated as well, but if it's binary I think we can treat it as buffer. Can you teach me how to get compile and use this plugin locally for my project? Glad to help!

jantimon commented 4 years ago

You don't have to compile anything just play around with node_modules/html-webpack-plugin/index.js

E.g. try to add a console.log statement :)

Just note that if you are using webpack-dev-server you will have to restart it in order to see any changes.

akphi commented 4 years ago

@jantimon yep, changing it to false works for me. Looks like addFileToAssets is only being used for favicon right now so it's probably safe to make this change. But my use case is utterly the most basic use case of this plugin, we probably should check with other folks who have more advanced use cases to see if this makes sense.

jantimon commented 4 years ago

Cool 👍 can you please create a pull request?

akphi commented 4 years ago

@jantimon do you want to expose the convertAsString flag as param for addFileToAssets or we just default it to false for now?

jantimon commented 4 years ago

not for now - if we will see that it causes any issue we can still add it

just set it to false

jantimon commented 4 years ago

@akphi released as html-webpack-plugin@5.0.0-alpha.7

dstarosta commented 4 years ago

When I run webpack with the --watch parameter, index.html file is not being generated. The version is 5.0.0-alpha.7.

jantimon commented 4 years ago

@dstarosta are you using the clean-webpack-plugin ?

dstarosta commented 4 years ago

@jantimon Yes. I found that the cache: false option fixes the problem.

jantimon commented 4 years ago

Cool I also added it to the todo list for the next major

jrnail23 commented 4 years ago

@jantimon: here's what I've found so far as I've tried upgrading to webpack5:

For reference sake, here's my html-webpack-plugin config:

    new HtmlWebpackPlugin({
      template: path.join(__dirname, 'src', 'views', 'index-template.html'),
      chunks: ['main'],
      inject: 'body',
      alwaysWriteToDisk: true,
      filename: 'index.html',
    }),
    new HtmlWebpackHarddiskPlugin({
      outputPath: path.resolve(__dirname, 'src', 'views'),
    }),

here's my (simplified) html template:

<!DOCTYPE html>
<html lang="en-us">
  <head>
    <meta charset="utf-8" />
  </head>
  <body>
    <div id="root" style="height: 100%"></div>
  </body>
</html>

and here's what I generally expect my html file to look like:

<!DOCTYPE html>
<html lang="en-us">
  <head>
    <meta charset="utf-8" />
    <link href="/public/main.f44825e38502163de35f.css" rel="stylesheet" />
  </head>
  <body>
    <div id="root></div>
    <script src="/public/client.bundle.main.b6f9288ceaebb62ab9b6.js"></script>
  </body>
</html>

first, we were on html-webpack-plugin@4.5 -- it actually seems to work just fine for us, although webpack 5 complains about a breaking change: (node:33423) [DEP_WEBPACK_COMPILATION_ASSETS] DeprecationWarning: Compilation.assets will be frozen in future, all modifications are deprecated. BREAKING CHANGE: No more changes should happen to Compilation.assets after sealing the Compilation. Do changes to assets earlier, e. g. in Compilation.hooks.processAssets. Make sure to select an appropriate stage from Compilation.PROCESS_ASSETS_STAGE_*.
then I tried updating to html-webpack-plugin@5.0.0-alpha.7 -- it doesn't work, as it writes the wrong bundle file to my html template (it writes the wrong hash) -- from my config: filename: 'client.bundle.[name].[contenthash].js',. Otherwise, the html is fine.
Beginning with html-webpack-plugin@5.0.0-alpha.8 and continuing to alpha.10, it seems to ignore my html template entirely. Here's an example of what actually gets written to my index.html file (note use strict getting written as content, which I assume is some incomplete portion of a bundle):
```
<head>
<link href="/public/main.9dbd8023185a251ff841.css" rel="stylesheet" /></head
>use strict
<script
defer="defer"
src="/public/client.bundle.main.7e131d3de7998ab8c3c8.js"
></script>
```

jantimon commented 4 years ago

@jrnail23 thanks for the detailed error report

I created a CodeSandbox of the html-webpack-plugin 5.0.0-alpha.10 html-webpack-plugin_5_x_alpha_-_CodeSandbox

Can you please create a fork and try to reproduce the errors you describe?

jrnail23 commented 4 years ago

I don't know, @jantimon, I feel pretty disrespected, what with you making me wait 14 whole minutes for a reply! (seriously kidding -- thanks for the really quick response!). I'll see what I can do with it.

jantimon commented 4 years ago

Your second point is really a big problem - the html-webpack-plugin injects the css & js before those files are being optimized

This was necessary to work around the deprecation message..

However in this compilation stage we don't know the final filename

In order to properly resolve that issue we need support from @sokra so I created an issue to find a better compilation stage here: https://github.com/webpack/webpack/issues/11822

Before that blocker is resolved we will probably not be able to release a stable html-webpack-plugin 5.x version

jrnail23 commented 4 years ago

@jantimon: so one thing I guess I left out: everything is fine when I run in watch mode, using devServer. My problems only occur when I run webpack CLI, in non-watch mode (production vs. development mode doesn't seem to matter).

I get what you're saying about the post-optimization issue... might that by chance be fixable via Webpack 5's new "true content hashing" (which, AFAICT, is intended to make contenthash reflect the source file content)?

Otherwise in the meantime, is there any indication that v4.5 is unsafe, despite the "breaking change" message?

jantimon commented 4 years ago

All unit tests are passing for 4.5 however I am not sure if it will work with future webpack 5 releases as it relies on a from now on unsupported webpack api

4.5 has also some issues with webpack 5s new singleton hook system.. so it will probably break if you have multiple webpack versions installed

jrnail23 commented 4 years ago

@jantimon, I wanted to draw your attention to this statement, which I added on after the fact:

My problems only occur when I run webpack CLI, in non-watch mode (production vs. development mode doesn't seem to matter).

Any thoughts on why it's (seemingly) bypassing my template in non-watch mode?

jantimon commented 4 years ago

Here is a working example with html-webpack-plugin@5.0.0-alpha.10:

webpack.zip

npm run webpack

jrnail23 commented 4 years ago

@jantimon, thanks, I'll take a look

jrnail23 commented 4 years ago

@jantimon OK, I think I finally have something useful for you here. Sorry these details have to drip out one at a time instead of giving you all the relevant stuff up front -- in trying not to throw the whole kitchen sink at you (and also to protect my employer's code), I oversimplified my examples.

We use express-es6-template-engine for our html pages.

Those files are saved as html files, and their content looks something like this (note the es6 template literal variable ${myTitle} for <title>):

<!DOCTYPE html>
<html lang="en-us">
  <head>
    <meta charset="utf-8" />
    <title>${myTitle}</title>
  </head>
  <body>
    <div id="root" style="height: 100%"></div>
  </body>
</html>

So my expected html file should look something like this (our express templating engine deals with that myTitle variable later, when it serves the page):

<!DOCTYPE html>
<html lang="en-us">
  <head>
    <meta charset="utf-8" />
    <title>${myTitle}</title>
  </head>
  <body>
    <div id="root" style="height: 100%"></div>
    <script defer="defer" src="main.js"></script>
  </body>
</html>

Since we end up including an html file in our output, we also add the following to our webpack config:

  module: {
    rules: [
      {
        test: /\.(html)$/,
        use: ['raw-loader'],
      },
    ]
  },

Now when I run webpack, I get the following in my dist/index.html file:

use strict<script defer="defer" src="main.js"></script>

Here is a minimal example: webpack.zip

It seems like raw-loader is not playing nicely with html-webpack-plugin. When I remove it, I get the correct output, but with it, I get the use strict output. I tried removing the raw-loader rule in my actual project as well, and it no longer has the use strict screwiness. It does, however, still have the wrong bundle hash, per the issue you already know about.

jantimon commented 4 years ago

@jrnail23 thanks it looks like the new webpack 5 child compiler is configured in a wrong way or buggy - I created a new issue here: https://github.com/webpack/webpack/issues/11909

jrnail23 commented 4 years ago

I'm glad you were able to make some sense of that. Thank you!

AviVahl commented 4 years ago

5.0.0-beta.11's typings.d.ts appears to be broken:

node_modules/html-webpack-plugin/typings.d.ts:36:13 - error TS2430: Interface 'Options' incorrectly extends interface 'Partial<ProcessedOptions>'.
  Types of property 'filename' are incompatible.
    Type 'string | ((entryName: string) => string) | undefined' is not assignable to type 'string | undefined'.
      Type '(entryName: string) => string' is not assignable to type 'string'.

36   interface Options extends Partial<ProcessedOptions> {

EDIT: fixed in beta.12

jantimon commented 4 years ago

Thanks @AviVahl I'll take a look

phungtuanhoang1996 commented 4 years ago

Hi @jantimon

Can I ask why are we hooking into after this stage PROCESS_ASSETS_STAGE_OPTIMIZE_SIZE instead of PROCESS_ASSETS_STAGE_OPTIMIZE_HASH? So that if filename uses contenthash it will be reflected in the html?

      compilation.hooks.processAssets.tapAsync(
        {
          name: 'HtmlWebpackPlugin',
          stage: webpack.Compilation.PROCESS_ASSETS_STAGE_ADDITIONS
          stage:
          /**
           * Hacky intermediate solution arround the open webpack issue https://github.com/webpack/webpack/issues/11822 which does not
           * allow to generate the HTML after the JS and CSS has been optimized
           *
           * This can break on any minor webpack release as it messes with the webpack internal enum values:
          */
          webpack.Compilation.PROCESS_ASSETS_STAGE_OPTIMIZE_SIZE + 200

https://github.com/jantimon/html-webpack-plugin/commit/73cb5f6d9f1b291786051f381dbc0ae2439802cd#diff-e727e4bdf3657fd1d798edcd6b099d6e092f8573cba266154583a746bba0f346R217

jantimon commented 4 years ago

Thanks @phungtuanhoang1996 for your feedback
You are right the html-webpack-plugin 5.x is not using the perfect timing to add the html assets

I couldn't find PROCESS_ASSETS_STAGE_OPTIMIZE_HASH in the official documentation: https://webpack.js.org/api/compilation-hooks/#additionalassets - is it save to be used?

In order to properly resolve that issue I also reached out to the webpack core team and created an issue to find a better compilation stage: https://github.com/webpack/webpack/issues/11822

@sokra has already started working on this topic and created a pull request https://github.com/webpack/webpack/pull/11956 to add a processAdditionalAssets assets stage but I still need to find out how it should be used in the optimal way - if you have any ideas please let me know

jantimon / html-webpack-plugin