Fetching remoteEntry.js file from a remote source on the server

[debugpai]

Hey everyone, First of all thanks a lot for the amazing work on module federation. I was wondering if it is possible to fetch the remoteEntry.js file from a remote source on the server. I know for client this is possible and was wondering if the same was possible on the server instead of fetching from the filesystem.

[debugpai]

Found out it is possible to do this. No special changes required. Also it is resolved at runtime so it would always get the fresh version of remoteEntry file

[ajayjaggi]

Hi @debugpai.Can you pls tell how to fetch on server side without filesystem?

[debugpai]

Hey @ajayjaggi. My bad. This feature is not possible on the server. From the server remoteEntry file can only be accessed from the filesystem. Unfortunately the open source part of module federation only solves for client side rendering properly.

[ZhengYuTay]

I guess it's possible to have a remote entry on the server as well, just need some workaround: Imagine there is two server, one provider(the one sharing the component) and one consumer server.

1. On your provider server, have a serverMiddleware that serve the remote server file.
2. On your server that consume the remote, have another serverMiddleware on that which request from the remote and build the remote file to local. Before this step, your build step might need to request one time and feed it to your webpack config.
3. The provider server can also provide a versioning on that end, so that if the version ever changed, the consumer can update its local file version and clear its cache on runtime.

This is a very backward thinking approach and I dont think it's a very good solution

[7rulnik]

It's possible. You can do it in this way:

   const remote = `new Promise(async resolve => {
  const content =  await someFetchFunction('http://hostname/appB/b-entry.js')
  const chunk = {}
  require('vm').runInThisContext('(function(exports, require, __dirname, __filename) {' + content + '\\n})')(chunk, require, 'some-fake-path', 'b-entry.js');
  resolve(chunk)
})`

    new webpack.container.ModuleFederationPlugin({
      remotes: {
        'b-entry': `promise ${remote}`
      },
    }),

Logic with require('vm') taken from webpack async-node target. Note that someFetchFunction should be defined somewhere in user-land code.

But it doesn't solve the problem at all because you will need a way to fetch remote chunks via HTTP. To do this you have 2 ways:

1. Teach webpack to fetch chunks over http. It's pretty risky and requires a custom chunk loading runtime. I will open-source it later. See https://github.com/webpack/webpack/issues/12084
2. Fetch and save remote chunks on same file system (or use some shared FS)

[estevaolucas]

Teach webpack to fetch chunks over http. It's pretty risky and requires a custom chunk loading runtime. I will open-source it later. See webpack/webpack#12084

This is probably what @jacob-ebey's WebpackNodeHttpChunkLoadingPlugin does here https://github.com/jacob-ebey/ssr-react-streaming-example/blob/main/ssr_react_streaming_example_a/webpack.config.js#L48-L50

[jacob-ebey]

@7rulnik pretty much nailed what the http chunk loading plugin does. There are some other changes needed in the webpack chunk loading runtime that's embeded into the remote entry otherwise it will try to load chunks from local disk instead of over the network.

[estevaolucas]

@jacob-ebey I have to ask it: by any chance do you have plans to open source WebpackNodeHttpChunkLoadingPlugin?

My problem with file system remote loading is: 1) can't have a distributed system for host and remotes. 2) I have to restart Next.js server (host) in order to get a fresh remote after making some changes on my remote (not sure if it's a FS issue - probably it's a webpack in-memory cache that I didn't figure out yet)

[jacob-ebey]

@estevaolucas I do not have plans at the moment, but with some of the other things I'm working on, it may come as part of those. Keep an eye on https://github.com/jacob-ebey/pwrc-react

[estevaolucas]

Cool. I'm watching PWRC's repo since you've announced it on your Twitter 😬

[a-h]

It was a total surprise to me that you can't fetch from the server-side. I didn't think it's clear in the (tiny amount) of documentation.

When I read the examples, I thought they had filesystem references just for convenience of writing the example, not that it just wasn't possible for them to be fetched remotely. The error message I got when I tried it didn't give the vaguest hint at the underlying problem, so I spent at least half a day thinking I was doing something wrong.

[flaviuse]

Hello, is there now an open source solution ?

[7rulnik]

You can check https://github.com/telenko/node-mf by @telenko

[tysonrm]

Check out https://github.com/module-federation/MicroLib/blob/master/webpack/fetch-remotes.js

On Jun 25, 2021, at 5:16 AM, Valentin Semirulnik @.***> wrote:

You can check https://github.com/telenko/node-mf https://github.com/telenko/node-mf by @telenko https://github.com/telenko — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/module-federation/module-federation-examples/issues/399#issuecomment-868394943, or unsubscribe https://github.com/notifications/unsubscribe-auth/AJI3W3QFAWJE6NOU5KS2RZ3TURJOHANCNFSM4SBSAZLQ.

[tysonrm]

There is an official streaming solution for node but its proprietary at the moment.

On Jun 25, 2021, at 5:16 AM, Valentin Semirulnik @.***> wrote:

You can check https://github.com/telenko/node-mf https://github.com/telenko/node-mf by @telenko https://github.com/telenko — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/module-federation/module-federation-examples/issues/399#issuecomment-868394943, or unsubscribe https://github.com/notifications/unsubscribe-auth/AJI3W3QFAWJE6NOU5KS2RZ3TURJOHANCNFSM4SBSAZLQ.

[ScriptedAlchemy]

We have multiple streaming solutions but only the worst will be publicly available, our older tech will be commercial and our latest innovation is patented and will require licenses or utilization of a SaaS