Webpack module federation breaks with multiple entrypoints

[codepunkt]

This time it's @sokra who told me this is probably the place to open an issue.

• Operating System: Windows 10
• Node Version: 12.16.2
• NPM Version: 6.14.4
• webpack Version: 5.0.0-beta.22
• webpack-dev-server Version: 3.11.0
• Browser: Chrome 84.0.4147.105, Firefox 79.0, Edge 84.0.522.48

• [x] This is a bug

Code

https://github.com/codepunkt/module-federation-examples/tree/dynamic-host-remote

Expected Behavior

Using webpack-dev-server instead of webpack should still support module federation with additional content tacked onto the remoteEntry by defining it as an additional entry.

Actual Behavior

Running "yarn start" to start webpack-dev-server breaks module federation and thus breaks the app in development mode.

For Bugs; How can we reproduce the behavior?

1. clone repository
2. ensure you're on branch "dynamic-host-remote"
3. run yarn on repo root
4. go into "dynamic-host-remote" directory
5. run yarn start in "dynamic-host-remote" directory
6. open localhost:3001 in the browser
7. encounter an error in the browser console that happens when executing app2/remoteEntry.js with the additional contents that were added to this entrypoint by webpack-dev-server
8. OPTIONAL: run yarn build && yarn serve and revisit localhost:3001 to see production build working just fine.

[alexander-akait]

@webpack-bot move to webpack/webpack-dev-server My mistake

[alexander-akait]

Feel free to send a fix

[codepunkt]

If I had a clue what the problem is, i'd send a PR right away. Spent a few hours with this, don't have the slightest idea 🤔

[alexander-akait]

/cc @sokra Any ideas on this?

[ScriptedAlchemy]

Okay, so the issue here is that WDS is likely appending its own entrypoint to the array / injecting a "require" to the hmr client. That typically serves as the entry module of an application. When ModuleFederation emits a remoteEnty with the same name as the entrypoint, they are merged.

This lets us attach initialization code, to federated remotes. something like changing publicPath.

webpack_public_path = new URL(document.currentScript.src).origin + "/";

https://github.com/module-federation/module-federation-examples/pull/277

The remote becomes 400kb big (usually like 5kb) and the module returned to the global is not __webpack_require__("webpack/container/entry/app1"); - therefore window.app1 will not be __webpack_require__("webpack/container/entry/app1"); and the federated API is never set to the global

@sokra i believe that either the WDS entry needs to return the container - which may lead to HMR capabilities? or the WDS entry module should not be applied to federated entry points when combined.

module.exports = {
  entry: {
    app1: "./src/setPublicPath",
    main: "./src/index",
  },
plugins: [
    new ModuleFederationPlugin({
      name: "app1",
      remotes: {
        app2: "app2@http://localhost:3002/remoteEntry.js",
      },
      shared: { react: { singleton: true }, "react-dom": { singleton: true } },
    }),
    new HtmlWebpackPlugin({
      template: "./public/index.html",
    }),
  ],

[sokra]

The wds entrypoint should be prepended instead of appended. The last value is exported.

[ScriptedAlchemy]

Will take a look, thanks @sokra

[ScriptedAlchemy]

Looks like we want to modify this

 const prependEntry = (originalEntry, additionalEntries) => {
      if (typeof originalEntry === 'function') {
        return () =>
          Promise.resolve(originalEntry()).then((entry) =>
            prependEntry(entry, additionalEntries)
          );
      }

      if (typeof originalEntry === 'object' && !Array.isArray(originalEntry)) {
        /** @type {Object<string,string>} */
        const clone = {};

        Object.keys(originalEntry).forEach((key) => {
          // entry[key] should be a string here
          const entryDescription = originalEntry[key];
          if (typeof entryDescription === 'object' && entryDescription.import) {
            clone[key] = Object.assign({}, entryDescription, {
              import: prependEntry(entryDescription.import, additionalEntries),
            });
          } else {
            clone[key] = prependEntry(entryDescription, additionalEntries);
          }
        });

[sokra]

It already prepends... So there is somethings else wrong

[sokra]

The bug is here:

https://github.com/webpack/webpack-dev-server/blob/4ab1f21bc85cc1695255c739160ad00dc14375f1/lib/utils/addEntries.js#L86-L92

Desprite the function name prepend it actually appends...

Change it to:

      /** @type {Entry} */
      const entriesClone = [].concat(originalEntry);
      additionalEntries.forEach((newEntry) => {
        if (!entriesClone.includes(newEntry)) {
          entriesClone.push(newEntry);
        }
      });

Could somebody send a PR for that? As test case you can use output.library = "TEST" and check if this created the correct global.

[alexander-akait]

Great, feel free to send a PR

[snitin315]

I will send a PR.

[codepunkt]

I'm not sure this will actually fix HMR with module federation - but i'm looking forward to try!

[ScriptedAlchemy]

Opening PR, will need to work on / help on test case

[knagaitsev]

I looked into this more closely, and here is the full problem:

In most cases, the dev server is trying to add some variation of these 2 entries (unless you disable their injection): webpack-dev-server/client/default/index.js and webpack/hot/dev-server.js.

Current behavior (webpack 5):

Our config has a single entry: main.js.

The 2 dev server entries are appended to the entry list (as explained above), resulting in something like:

main: {
  import: {
    'main.js',
    'webpack-dev-server/client/default/index.js',
    'webpack/hot/dev-server.js'
  }
}

Then we call the entryOption hook with all of these entries here: https://github.com/webpack/webpack-dev-server/blob/4ab1f21bc85cc1695255c739160ad00dc14375f1/lib/utils/updateCompiler.js#L51

This results in an entry list on the webpack side that looks like this: main.js, main.js, webpack-dev-server/client/default/index.js, webpack/hot/dev-server.js

So it would seem that prepending the 2 entries would fix the issue: main.js, webpack-dev-server/client/default/index.js, webpack/hot/dev-server.js, main.js

However, it is not that simple because webpack essentially filters out the duplicate entries, taking only the first of the duplicates (https://github.com/webpack/webpack/blob/bdf4a2b942bed9d78815af828f7935ddfcd3d567/lib/Compilation.js#L1764), so we get: main.js, webpack-dev-server/client/default/index.js, webpack/hot/dev-server.js (regardless of if we append or prepend in the webpack step). The very last in this list is currently the one that is exported, so in no case is it ever main.js while there are other entries being injected.

Thus, the solution is to either change how this filtering works, or change the selection of entries by webpack such that the very first entry from the entryOption hook is used, rather than the very last.

I think the latter is preferred, because I realize now that it is not ideal to be pushing duplicate entries into the entryOption hook (as we do currently). Instead, we should only be pushing webpack-dev-server/client/default/index.js and webpack/hot/dev-server.js into the entryOption hook, as they have not yet been registered as entries. On the webpack side, these entries should not be considered as the module exports, as they were added later after the initial configuration entries.

[sokra]

Then we call the entryOption hook with all of these entries here:

https://github.com/webpack/webpack-dev-server/blob/4ab1f21bc85cc1695255c739160ad00dc14375f1/lib/utils/updateCompiler.js#L51

Why does webpack-dev-server do that? The hooks is not owned by webpack-dev-server. It should not call it. It's already called by webpack itself. That's probably also the reason why the entries are added twice...

Seems like this seem to be some kind of hack to reapply the entry option, because it has been modified after the options has been applied, which is too late. It doesn't make sense to modify the options after they have been applied to the compiler (converted to plugin). You need to use a plugin instead.

A plugin to add these entries for webpack-dev-server could be like that:

compiler.hooks.make.tapAsync({
    name: "webpack-dev-server",
    stage: -100
}, (compilation, callback) => {
    const options = {
        name: undefined // global entry, added before all entries
    };
    const dep = EntryPlugin.createDependency("webpack-dev-server/client/...", options);
    compilation.addEntry(context, dep, options, err => {
        callback(err);
    });
});

Note the hack works in webpack 4 as each entrypoint can only have a single module (arrays are wrapped in a artificial module), so reapplying the entryOption also adds it twice, but the second one overrides the first one.

There is no global entry in webpack 4, so the above plugin doesn't work for webpack 4. I would keep the hack as legacy code, as a clean solution is more complex.

[codepunkt]

Thanks everyone for chiming in.

What confuses me a little about this is that webpack-dev-server, which i always thought of as a "major part of webpack", actually seems to be an afterthought when it comes to new developments.

I don't understand a lot of the details everyone wrote in this issue, but i'm glad this problem is being ironed out and fixed.

[pmmmwh]

Note the hack works in webpack 4 as each entrypoint can only have a single module (arrays are wrapped in a artificial module), so reapplying the entryOption also adds it twice, but the second one overrides the first one.

There is no global entry in webpack 4, so the above plugin doesn't work for webpack 4. I would keep the hack as legacy code, as a clean solution is more complex.

@sokra is there a "correct"/"clean" way to inject global entries as a plugin (for Webpack 4)? I'm asking about this for fast-refresh in general because we need to ensure global runtime code is being setup before user code. Similar to WDS, what we're doing today is this (without the entryOptions hack) within apply:

https://github.com/pmmmwh/react-refresh-webpack-plugin/blob/main/lib/utils/injectRefreshEntry.js

[MaximSagan]

Sorry to be that guy, but... any updates?

[alexander-akait]

Fully work on the next releases (included stable release) will be started on the next week

[wangmeijian]

下一个版本（包括稳定版本）的全部工作将在下周开始

When will it be updated?

[csvan]

下一个版本（包括稳定版本）的全部工作将在下周开始

When will it be updated?

When it's done, feel free to help :)

[raix]

@alexander-akait are there anything we can help out with eg. code/test/debug etc. not sure how much is missing/done?

(Despite visiting issues/pr's opened/closed and trying to follow things merged to master etc. I still don't have an overview on how to help out here)

[alexander-akait]

@raix Here fix https://github.com/webpack/webpack-dev-server/pull/2920, can you try locally? I need rewrite some tests, if it will work I will focus on it and will do release, I already work on webpack-dev-server (focused on webpack-cli and webpack-dev-server, want to finish all problem to move ahead)

[robinlarsson]

@alexander-akait are there any updates around this issue? We would love to start using Module Federation and this fix of yours would be very useful. Are you still in need of testing for #2920 locally?

[csvan]

Now that #2920 is merged, I tried running with Module Federation locally, but it seems to fail here (in the entry point of the federated module):

function checkDeferredModulesImpl() {
    var result;
    for (var i = 0; i < deferredModules.length; i++) {
        var deferredModule = deferredModules[i];
        var fulfilled = true;
        for (var j = 1; j < deferredModule.length; j++) {
            var depId = deferredModule[j];
            if (installedChunks[depId] !== 0) fulfilled = false;
        }
        if (fulfilled) {
            deferredModules.splice(i--, 1);
            result = __webpack_require__(__webpack_require__.s = deferredModule[0]);
        }
    }
    if (deferredModules.length === 0) {
        __webpack_require__.x();
        __webpack_require__.x = x => {
        };
    }
    return result;
}

The reason is that the following line will always set fulfilled to false, because dev-server injects its own entries into deferredModule:

if (installedChunks[depId] !== 0) fulfilled = false;

Thus result will always be undefined.

[alexander-akait]

@csvan Can you provide example? I think we need fix it better

[csvan]

@alexander-akait not yet unfortunately, noticed it in one of our own projects when running dev-server with a Module Federation project. I could try setting a small repro up.

I wonder where this comes from though, I suppose it is added by Webpack:

var deferredModules = [

      ["./node_modules/webpack-dev-server/client/default/index.js?http://0.0.0.0", "vendors-node_modules_webpack-dev-server_client_default_index_js_http_0_0_0_0-node_modules_web-394369"],

      ["webpack/container/entry/supportwidget", "vendors-node_modules_webpack-dev-server_client_default_index_js_http_0_0_0_0-node_modules_web-394369"]
    ];

[csvan]

@alexander-akait repro here: https://github.com/csvan/wds-mf-repro

It needs the latest version of webpack-dev-server though, the repro references the last beta

[alexander-akait]

Thanks, I will look at this in near future, the next beta will be on this week, so I try to resolve it

[csvan]

Dug a bit more, this only seems to happen when this evaluates to true during the build:

https://github.com/webpack/webpack/blob/80c5041fe1f909d34975abd33e03bdb069876ea9/lib/web/JsonpHelpers.js#L61

As long as the checkDeferredModulesImpl is not added to the entry point by webpack, things work well on master - however, the problem is still that webpack-dev-server breaks checkDeferredModulesImpl by adding additional entries which cause it to return undefined.

[kumarlachhani]

Request @alexander-akait to help us in updating about the issue.

[alexander-akait]

It think should be fixed in beta-1, somebody can try?

[kumarlachhani]

Sorry for asking this again but any updates on this issue? @alexander-akait bro?

[alexander-akait]

@kumarlachhani Can you try https://github.com/webpack/webpack-dev-server/blob/master/package.json#L3? I think should work

[mawi12345]

I have setup a minimal example (just module-federation dynamic-remotes with updated dependencies) and hot reloading is working with module federation. Not sure if this is exactly what @codepunkt reported.

[ruanyl]

@mawi12345 I tried your examples, the hot reload works if you open the host app http://localhost:3001/, but it doesn't work if you open http://localhost:3002/ directly.

The warning shows up:

[HMR] Update failed: ChunkLoadError: Loading hot update chunk main failed.
(missing: http://localhost:3002/main.4ad69454e89bc241f658.hot-update.js)
    at http://localhost:3002/main.js:1200:26
    at new Promise (<anonymous>)
    at loadUpdateChunk (http://localhost:3002/main.js:1195:20)
    at http://localhost:3002/main.js:1655:29
    at Array.forEach (<anonymous>)
    at Object.__webpack_require__.hmrC.jsonp (http://localhost:3002/main.js:1650:22)
    at http://localhost:3002/main.js:828:45
    at Array.reduce (<anonymous>)
    at http://localhost:3002/main.js:824:53

[alexander-akait]

@ruanyl Why it should work on http://localhost:3002/?

[mawi12345]

@mawi12345 I tried your examples, the hot reload works if you open the host app http://localhost:3001/, but it doesn't work if you open http://localhost:3002/ directly.

@ruanyl you are correct, hot reloading is not working on the standalone + remote apps only on the host app. Not sure if this issue is about hot reloading the standalone remote.

[ruanyl]

@mawi12345

Alright, I got some findings.

The reason is the HtmlWebpackPlugin takes the main chunk and the remoteEntry.js chunk. If I open the app directly as a host app, remoteEntry is not needed.

So I configured HtmlWebpackPlugin to only include the main chunk, and this solved the problem:

      chunks: ["main"],

Now webpack hot works.

[ScriptedAlchemy]

Should open a bug on html webpack. Tho I thought they fixed this in their v5 of the plugin

[omkar-nath]

Hi, We are also facing this issue. When we try to inject remoteEntry.js using System.js in our index.html for dev mode, HMR stops working. If we directly inject the main.js file in index.html, it works .

[ScriptedAlchemy]

I'm coming out with a life reload plugin that kinda attempts to solve the problem. Not hot reloading but it'll refresh the change when making them to other containers

[craigmiller160]

Hi. Just chiming in. I'm using webpack 5.50.0, html-webpack-plugin 5.3.2, and webpack-dev-server 3.11.2 and I am experiencing the issue previously described. When using module federation, I get the errors from the OP. These errors only started occurring once my app was consuming at least one remote, if that helps at all. Wondering what can be done, what you might need from me to make more progress?

[tianyingchun]

HMR not working with multiple entries i upgrade webpack-dev-server from v4.0.0-rc.1 to v4.0.0 it will infinite loop for update.json 404. i use "webpack": "^5.50.0", BTW: webpack-dev-server@4.0.0-rc.1 with webpack@^5.50.0it work fine.webpack-dev-server@4.0.0 with webpack@^5.50.0 was broken, infinite loop for update.json

[alexander-akait]

@tianyingchun Can you provide reproducible test repo?

[tianyingchun]

1. install webpack@^5.50.0, webpack-dev-server@4.0.0

2. configuration webpack with two entries or above

3. while start serve mode with hot configuration

   hot: true,
     compress: true,
     open: false,
     port: mockPort,
     static: {
       directory: `${join(projectCwd, '/public')}`,
     },
     // Load https
     https: evolveOptions.https,
     // Enable firewall or set hosts that are allowed to access the dev server.
     allowedHosts: 'all',
     client: {
       overlay: {
         errors: true,
         // don't show overlay warning message in dev page, view warning mesage in console.
         warnings: false,
       },
     },

   while page loading first it will infinite loop hot reload.

4. while update really code related one of entry , this entry is reload that's ok

5. for single entry works fine both rc.1 and v4.0.0

Totally this issue occurs only while first loading of webpack serve.

it works ok once we have some really code modified, and triggered really dev-server reload

[tianyingchun]

i think in client-src/utils/reloadApp.js it has something broken while dev-server initial

unction reloadApp({ hot, liveReload }, status) {
  if (status.isUnloading) {
    return;
  }

  // TODO Workaround for webpack v4, `__webpack_hash__` is not replaced without HotModuleReplacement plugin
  const webpackHash =
    // eslint-disable-next-line camelcase
    typeof __webpack_hash__ !== "undefined"
      ? // eslint-disable-next-line camelcase
        __webpack_hash__
      : status.previousHash || "";
  const isInitial = status.currentHash.indexOf(webpackHash) === 0;

  if (isInitial) {
    const isLegacyInitial =
      webpackHash === "" && hot === false && liveReload === true;

    if (isLegacyInitial) {
      status.previousHash = status.currentHash;
    }

    return;
  }

  function applyReload(rootWindow, intervalId) {
    clearInterval(intervalId);

    log.info("App updated. Reloading...");

    rootWindow.location.reload();
  }

  const search = self.location.search.toLowerCase();
  const allowToHot = search.indexOf("webpack-dev-server-hot=false") === -1;
  const allowToLiveReload =
    search.indexOf("webpack-dev-server-live-reload=false") === -1;

  if (hot && allowToHot) {
    log.info("App hot update...");

    hotEmitter.emit("webpackHotUpdate", status.currentHash);

    if (typeof self !== "undefined" && self.window) {
      // broadcast update to window
      self.postMessage(`webpackHotUpdate${status.currentHash}`, "*");
    }
  }

[alexander-akait]

I will look at this in near future, but nothing special here, you should not get infinity reload if you have hot: true

[tianyingchun]

i have tried, replace all client/index.js from rc.1 it works fine.