[Feature Request]: Support module federation

[zackarychapple]

What problem does this feature solve?

Module Federation is heavily used within many large organizations and is going to become a blocker for many organizations moving to rspack.

Module Federation support is listed as one of the required examples. Some of the potential issues are highlighted in this issue around custom rspack runtime.

What does the proposed API of configuration look like?

There are several examples already existing in module-federatin-examples that can be duplicated / enhanced to run with rspack in addition to webpack. These may not all be solvable at once however the below examples would be ideally supported together with module federation in rspack.

• [ ] CRA example
• [ ] React + HMR example
• [ ] React + Delegate Modules example
• [ ] Rust + Wasm example
• [ ] Vue example
• [ ] Next v13 example

[h-a-n-a]

There are some missing features related to module federation:

• [x] Library-type support: https://webpack.js.org/configuration/output/#outputlibrarytype option var seems to be used a lot and by default.
• [ ] compiler hook afterPlugins: Module federation plugin leverages this to register its sub-plugins.
• [ ] Some runtime globals are not currently supported yet, here: https://github.com/webpack/webpack/blob/b67626c7b4ffed8737d195b27c8cea1e68d58134/lib/container/ContainerReferencePlugin.js#L130-L134, and here if you want to support shared module: https://github.com/webpack/webpack/blob/b67626c7b4ffed8737d195b27c8cea1e68d58134/lib/sharing/ConsumeSharedPlugin.js#L302-L307
• [ ] There are some other NormalModule hooks missing, but most of them lie in the plugin for shared module.
• [ ] A custom module factory is hardcoded for now: https://github.com/modern-js-dev/rspack/blob/537b447fa288dd2c6e50aa96985bddbf24974199/crates/rspack_core/src/compiler/queue.rs#L75

Features that supported and useful for mf:

1. Dependency: You can implement your own dependency struct with trait Dependency: https://github.com/modern-js-dev/rspack/blob/537b447fa288dd2c6e50aa96985bddbf24974199/crates/rspack_core/src/dependency/mod.rs#L70, example for a simple CssUrlDependency: https://github.com/modern-js-dev/rspack/blob/537b447fa288dd2c6e50aa96985bddbf24974199/crates/rspack_core/src/dependency/css/url.rs#L8
2. factorize for normal module factory: A simple example can be found here: https://github.com/modern-js-dev/rspack/blob/537b447fa288dd2c6e50aa96985bddbf24974199/crates/rspack_plugin_externals/src/plugin.rs#L22
3. Module: The trait for module can be found here: https://github.com/modern-js-dev/rspack/blob/537b447fa288dd2c6e50aa96985bddbf24974199/crates/rspack_core/src/module.rs#L37

Implementing non-shared module federation looks promising at the moment, and don't need to tweak a lot.

[zhoushaw]

@zackarychapple @h-a-n-a Hello everyone, I have a question about how Rspack will support Module Federation. Will it implement this feature using a Rust code plugin in the crates or will it support the JavaScript's module federation plugin? The reason I ask is because at Bytedance, we have implemented a Vmok scheme that is not solely based on module federation.

With Vmok, we have split the bundler and remote logic in order to make it possible for consumers to use remote modules without having to access the bundler plugin. This allows consumers to have greater control over the load process, for example, by adding remote module load retry or log. However, Vmok's split chunk logic is based on dependencies, factorization, and module, which is similar to Module Federation.

If Module Federation must be implemented in Rspack's internal plugins, it means that we cannot customize the Module Federation plugin.

[zackarychapple]

@zhoushaw, that is a really good question. Right now module federation does not work with the existing plugin because the underlying hooks are not yet available in Rspack. Since we can use the JS plugin with Rspack it would not need to be migrated to Rust immediately. Will schedule some time to talk with you, @h-a-n-a, and @hardfist to best support MF + Vmok on Rspack.

[ScriptedAlchemy]

Federation is built ontop of 3 core aspects.

For containers (remotes) we use EntryPlugin - to create a special entrypoint that can either clone the webpack runtime functions into it, or if you have runtimeChunk: single, it will reference a separate chunk and share the module graph.

Container plugin depends on the output format, var is the default which creates a global variable. Theres also self, global, window, module and any other output formats one would expect from a normal webpack main.js output.

ContainerReferencePlugin is an Augmentation of ExternalsPlugin, it how hosts consume remotes. It too can specify remoteType to better understand what the consuming remote is, (umd, window, assign, var)

Since its build ontop of externalsPlugin, we had to rewrite externals significantly, to support async externals like an external type "script" which tells webpack hosts how to bind to, or inject an external.

This is also how the @ syntax was created, and later was pulled upwards for use in externals independently.

[global@url, subreference, exportName] for instance lets webpack know to inject a script and bind to the global - then we can point it at a specific object on that global property - if need be.

The last part was the more complex aspect - which is SharePlugin (ProvideSharePlugin,ConsumeSharePlugin) which hooks into chunking logic to register any shared module as an AsyncDependencyBlock

SharePlugin adds additional runtime modules to webpack host and remote for registering scope, semver matching, and choosing if it should loadFallback or reference another module already in registered in the scope.

If you share a module multiple times, webpack can prefer one within semver thats already loaded, theres markers on shared modules such as loaded:1

Regarding customization to module federation - i built the concept of "delegate modules" recently, which allow me to control the glue code of module federation in any way i require - with delegate modules we most likely can achieve anything one would usually modify the core plugin to do. One reason we do not provide any hooks to federation is to ensure consistency and reliability between the interfaces. the {get,init} object is the main constant, but one could enable additional properties on the object - but required adjusting the runtime modules to do so accordingly.

https://github.com/module-federation/module-federation-examples/blob/master/nextjs-ssr-delegate-modules/checkout/remote-delegate.js

https://github.com/module-federation/module-federation-examples/blob/master/nextjs-ssr-delegate-modules/checkout/next.config.js#L7

[2heal1]

@ScriptedAlchemy , thanks for sharing MF core aspects ! Since @zhoushaw ask the question , I add some context about why Vmok extract the runtime:

1. User can load MF Module and use shared dependency by using runtime sdk only ,not need to use bundler plugin .
2. User can know more clearly about whether the shared module has take effect , and user can know which shared has used in which remote module.
3. Through extracting the runtime ,we can do more extending thing by plugin system, such as preload feature.

Back to the question , Rspack not provide native plugin or js plugin (support process module and dependency) yet . And if we want to implement MF ,we may develop internal in Rspack .

Once we migrate MF to Rspack(not extract runtime) , it will be difficult for Vmok to customize feature on Rspack.

Regarding above question ,I thought of two solutions:

1. Will MF extract runtime and supports other customization ?
2. Rspack exposes the processing logic of Module and Dependency on the js side, not sure about the impact on performance

[2heal1]

After learning "delegate modules" . It seems that the runtime is still webpack runtime which may not easy to do some extending thing on outside.

[zhoushaw]

Can I implement the module federation feature?

[ScriptedAlchemy]

I'd need to understand better what capabilities you need from ' within module federation that require extensions.

I've got a decent amount of experience in its design and adaptability, some extensibility has always been an area I'd like to see a little improvement in - but I'd preserve the base get/init interface and see how we can offer additional interfaces or more "middleware" type patterns within existing get init. Delegate modules has provided a lot of flexibility on consumer side, and we can add startup code that merges with remote container to offer other capabilities as well. Usually a combo of using other parts of webpack in specific ways that leverage how underlying mechanisms behave.

I've been able to get good mileage out of its interfaces and have mostly not encountered hard limits on its extensibility. For my most advanced users and cases, the extensions are usually around runtime modules such as chunk loading, or how I enabled mfp in edge network or node.js. However there are some refinements I believe could be introduced which increase its power more directly.

Let's fully understand the needs and limitations, then explore the best way to implement. Tobias has several reasons for its design & we had to make large changes to webpack. I'd want to ensure we do not inadvertently introduce a pattern that was deliberately avoided during iterations of webpack 5 betas.

Understanding vmok requirements would help as I initially started with mostly using the runtime sdk as well. I primarily only used module federation without much or any use of the plugin. Over time I've improved tooling and strategy for runtime sdk and other runtime modules

How I've integrated delegates has allowed for extraction and still using runtime sdk, using delegate and module hoisting, you can still use the dynamic abilities , but still leverage other runtime modules in a elegant but powerful manner.

[zhoushaw]

Thank you for your detailed reply. Our idea will not disrupt the overall structure of Module Federation, but rather we only hope to control the process within it. I will supplement our expectations and user API usage, and provide use cases to help you better understand our demands. Here are our goals and pseudocode:

We hope that developers can dynamically add remote modules instead of declaring all references in the build plugin (Why do we want to do this? In our business scenario, there are many users from different teams, each responsible for developing different SPA route subpages. They can dynamically load corresponding submodules when entering different subroutes. They prefer to configure the module list directly on our deployment platform to add new pages within a system, but the current module federation has a fixed module list).

Moreover, for some systems, it is difficult to add build plugins and change usage methods in their applications, which prevents them from benefiting from remote modules. With our abstraction, users can use our solution without relying on build plugins and can also combine or use them alone if needed.

// The following is pseudocode and cannot be run directly
// Runtime code section
import React from 'react';
// This part of the code can run in any environment, even if the user does not use the build plugin, it can still run. When used in conjunction with the build plugin, it becomes more flexible and can provide features such as dependency reuse.
import { init, loadRemote } from '@vmok/kit/runtime';

// Use default export
// In fact, import will eventually be converted to loadRemote
const App3 = React.lazy(() => import('remote3'));

// Get the remote module list
async function getRemotes() {
    return new Promise((resolve) => {
        setTimeout(() => {
            resolve([
                {
                    name: 'remote1',
                    entry: 'http://xxx/remoteEntry.js'
                },
                {
                    name: 'remote2',
                    entry: 'http://xxx/remoteEntry.js'
                }
            ]);
        }, 2000);
    });
}

export default function App() {
    const [remotes, setRemotes] = useState([]);
    useEffect(() => {
        getRemotes().then((remotes) => {
            setRemotes(remotes);
            init({
                remotes
            });
        },[]);
    });
    return (
        <BrowserRouter basename={"/"}>
            <Routes>
            <Route path="/" element={<App store={store} />}>
                {
                    remotes.map((remote) => {
                        const Component = React.lazy(() => loadRemote(remote.name));
                        return <Route path={`/${remote.name}`} element={<Component />} />
                    })
                }
                <Route path="/app3" element={<App3 />} />
            </Route>
            </Routes>
        </BrowserRouter>
    )
}

// Build plugin section
const { VmokPlugin } = require('@vmok/kit/webpack-plugin-v5');

// webpack plugins
// The build plugin will generate a module protocol for other modules to use based on shared
// The generated module protocol is completely consistent with module federation
module.exports = {
    plugins: [
        new VmokPlugin({
            name: 'app',
            // Only remote3 is provided, and remote1 and remote2 can be used at runtime. The build plugin can be combined with runtime
            // Because the build plugin directly references the runtime API
            remotes: {
                remote3: 'remote3@http://xxx/remoteEntry.js',
            },
            shared: ['react', 'react-dom']
        })
    ]
}

2. We hope to collect more information about the SDK's invocation process at runtime, such as the loading time of shared and which shared components were loaded by whom.

// The following is pseudocode and cannot be run directly
import { init, Plugin } from '@vmok/kit/runtime';

function collectSharedInfo(): typeof Plugin {
    return {
        name: 'vmok-collect-shared-info',
        loadShared(sharedInfo) {
            // sharedInfo: who provides shared, who loads it
            // We can collect shared information, store it in our specific data structure
            // And provide it to our Chrome plugin, allowing users to visualize their shared reuse and loading time situations for performance optimization
        },
        // Users can define their own hooks for loading failure, which can fallback to their own resources or retry, and so on.
    }
}
init({
    plugins: [
        collectSharedInfo()
    ]
});

Our current splitting is more like the relationship below.

[zhoushaw]

However, such an idea may not necessarily be applicable to the current community, but for the scenarios we are facing, we hope to provide such an API for the convenience of other developers.

Our current idea is that, in the Rspack build implementation, we will abstract the init, loadRemote, and loadShared APIs, and then allow Rspack to support an additional parameter when running. In our company's internal projects, we can replace the reference source so that the runtime references our internal API. This way, we can connect with our infrastructure on this basis.

// rspack.config.js
export default {
  builtins: {
    federation: {
        // Parameters and functions are completely consistent with the webpack federation plugin
        name: 'xxx',
        remotes: {},
        shared: {},
        // Only add an additional optional parameter
        runtime: require.resolve("@vmok/kit/runtime")
    }
  }
}

In summary, although this idea may not be applicable to the broader community, it is useful for the scenarios our are facing. By abstracting init, loadRemote, and loadShared APIs in the Rspack build implementation and allowing Rspack to support an additional parameter when running, I can replace the reference source so that the runtime references my internal API. This approach will enable our to connect with our infrastructure more seamlessly.

[brunos3d]

Wow, I'm impressed with this discussion folks. This is a great step toward making Module Federation a bundler abstract/isomorphic tool :rocket:

[ScriptedAlchemy]

Okay, I believe that there may be ways to improve your needs. I do support this type of dynamic approach and some of delegate modules and internal node plugin uses low level api.

I also have ways to variable import which lets me skip the low level api, but get the same resulting flexibility.

If rspack supported(remote-engine/${someVar}) then we could dynamically load remotes within webpack import mechanisms but without any hardcoding.

What I'm saying is yes to your use case, but I believe we could improve upon how to interface dynamically. We are aligned, I just want to show you my ideas and hear more about your needs to see if anything I've worked on would be a tactical advantage to improve your options

[wrick17]

Any progress on this?

[zackarychapple]

There is some refactoring being done before resuming work on MF support, however a lot of the hooks needed AFAIK were already added when adding Angular support. More will be knocked out as we work on NextJS support.

[ScriptedAlchemy]

I tried adding the webpack plugin to rspack config just by direct importing it from webpack, natrually that didnt work but gave me some insight. Ill circle back to this once ive spoken with a few members of the infra team, in the meantime ill start setting up a rspack equivalent reference architecture (probably using modernjs as the base), similar to the one i had for next.js federation.

[zackarychapple]

@viktoriialurie should keep an eye on this so we can keep it on the doc site / roadmap.

[pratyushseq]

@ScriptedAlchemy Any progress on this? Eagerly waiting for this 😋

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. If this issue is still affecting you, please leave any comment (for example, "bump"). We are sorry that we haven't been able to prioritize it yet. If you have any new additional information, please include it with your comment!

[krassowski]

Bump. Downstream, this would be necessary for adoption in Jupyter (https://github.com/jupyterlab/jupyterlab/issues/15035).

[hardfist]

we're planning to working on Module Federation Support this quarter. we will post our detail plan soon

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. If this issue is still affecting you, please leave any comment (for example, "bump"). We are sorry that we haven't been able to prioritize it yet. If you have any new additional information, please include it with your comment!

[hardfist]

It's supported in 0.5 version, see the release blog https://www.rspack.dev/blog/announcing-0.5