ScriptedAlchemy commented 4 years ago

Feature request

@Timer @rauchg @timneutkens In order to stop my "Twitter Driven Development" (still love that quote) I have updated the description of this issue to hopefully help move this challenge forwards.

Is your feature request related to a problem? Please describe.

Support Module Federation ability to share vendor code, like react

Describe the solution you'd like

We should not use entrypoint based routing, each entrypoint would require react and react-dom to be async imported, or each entrypoint would need to use the import(./bootstrap) method in order to allow webpack to figure out if react exists already or if it needs to go and download the chunk. Ideally, there should be only one entrypoint, then whatever else needs to happen is done so via code splitting and async imports.
An alternative nice to have would be to make shared: {import:'react',eager:true} work - this would force react to be loaded upfront - however it seems that react is included multiple times in each entrypoint - causing various multiple copies of react issues. If eager worked, I could at least share react and force the remote to always use the hosts copy of react (I think) - this would / should be okay since next mostly requires us to be on a somewhat consistent version of react. Like next 11 will use v17, since all my apps use the latest nextjs, they will be compliment with a v17 version of react.

The bare minimum to get this working client or server-side would be to have some flag that changes how the client entrypoint gets started. If the client.js looked like this https://github.com/module-federation/module-federation-examples/blob/master/bi-directional/app1/src/index.js then there's a high likelihood we could hold up the application until whatever shared modules with static imports were ready before hydration begins.

Current workaround

My workaround no longer works for SSR (since 10.x), but works for CSR - is to alias react as some external global https://github.com/module-federation/nextjs-mf/blob/main/react.js

~I then patchSharing which is an ultra hack that literally inlines react into the head of _document as a script https://github.com/module-federation/nextjs-mf/blob/main/patchSharing.js~ I use resolve.alias against react$ to point react imports to my custom react file https://github.com/module-federation/nextjs-mf/blob/main/withModuleFederation.js#L27

~This is not optimal for production-grade use but the limits of next give us little choice~ Seems okay for production grade apps

Even with Webpack 5, what we have done to get our federated AB testing, Tag manager, and AB testing engine to work with next is apply a shim that creates a fake sharing API that attaches react to share scope manually. This mechanism is the same one that I used a year ago when next was still webpack 4 based and I was using webpack 5 remotes against v4 hosts.

import logger from "../logger";
/*  This logic is needed because we're not using webpack 5 yet */

const sharedExports = {};

const shareExports = (exports, remote = "") => {
  /*
  exports is expected to be an object
  {
    "react": ()=> Promise.resolve(()=>require("react"))
  }

  The value should be:
  - a function that returns a promise
  - that promise should resolve to a function
  - that function should return the module
  */

  // if no remote is given "" is used and that will apply to all remote entries
  // This method is additive, there is no "unshare"
  sharedExports[remote] = Object.assign(sharedExports[remote] || {}, exports);
};

const applySharedExports = (remote) => {
  if (typeof window !== "undefined") {
    // Given a remote entry, we gather all the modules that the host application has already
    const mergedExports = {
      ...sharedExports?.[""], // apply globally shared modules
      ...sharedExports?.[remote] // modules shared only for this remote entry
    };

    // Logic for applying @ScriptedAlchemy  
    // check if using new webpack API
    const override = window?.[remote]?.init
      ? window?.[remote]?.init
      : window?.[remote]?.override;
    if (window?.[remote]?.init) {
      Object.assign(
        mergedExports,
        Object.entries(mergedExports).reduce((acc, [key, value]) => {
          if (typeof value === "function") {
            let version = "16.13.1";
            try {
              // eslint-disable-next-line
              version = value.version;
            } catch (error) {
              logger.error("Error retrieving shared version", error);
            }

            Object.assign(acc, {
              [key]: {
                [version]: {
                  get: value,
                  loaded: true,
                  from: "dar"
                }
              }
            });
          }
          return acc;
        }, {})
      );
    }
    if (override) {
      try {
        override(Object.assign(mergedExports, __webpack_require__.O));
      } catch (error) {
        logger.warn(`Unable to apply webpack 5 shim of shared exports`, error);
      }
    }
  }
};

export { shareExports, applySharedExports };

ScriptedAlchemy commented 3 years ago

~~Update on this.~~

~I've found a way to move the async boundary into the webpack runtime. This circumvents limitations of next.js by changing how webpack startup runtime module works.~

~Replacing the startup module will effectively prevent next from initializing till webpack container negotiations take place.~

~I only uncovered this today, so it'll take some time to modify and PR webpack.~

Too hard

ckken commented 3 years ago

mark

ScriptedAlchemy commented 3 years ago

@rauchg @timneutkens @Timer @sokra I've updated the git issue, please let me know if there's any more information I can provide

mlstubblefield commented 3 years ago

Awesome! We're super excited about utilizing next+module federation to support microfront ends.

And for what it's worth (seems like everyone has a different definition of micro front ends) here's the short definition of what we're wanting to do.

We want to be to deploy our header/footer (layout) as a next app in vercel as well as 3-4 pages as separate next js apps in vercel.

Ideally they're stitched together and client side routing is well supported (hopefully the nextjs libraries are shared between the apps).

ScriptedAlchemy commented 3 years ago

This is more or less what im looking for as well, seamless page routing & component level ownership ☝️

ckken commented 3 years ago

mark

ckken commented 3 years ago

mark

ScriptedAlchemy commented 3 years ago

I've gotten Module Federation and Next routing to work. https://twitter.com/ScriptedAlchemy/status/1402319022331502592?s=20

This works client-side and probably could work server-side with some tweaks.

You'll need a catchall page and the nextjs-mf plugin.

Sample PR: https://github.com/module-federation/module-federation-examples/pull/859

Likely ill move to a switch statement so I can take advantage of the import() syntax over the low-level API

Also will probably use a suspense implementation and if need be, write some utility which will be added to next-mf if a well-rounded solution looks cumbersome

@mlstubblefield ☝️

ScriptedAlchemy commented 3 years ago

As of today, I was able to use just the Module Federation plugin with next js.

It appears that sharing works now, at least when it's in eager mode.

~As of my testing today - nextjs supports federation without any workarounds~. 🎉

It's almost too good to be true.

I'll perform some deeper tests tomorrow to confirm.

Darth-Knoppix commented 3 years ago

I’m having mixed results getting a good setup with Next.js 11 due to the issues @ScriptedAlchemy describes. Does anyone have an example project using Next.js 11 bi-directional CSR?

ScriptedAlchemy commented 3 years ago

you have to use the next-mf plugin, which pretty much makes a third webpack build

jonniedarko commented 3 years ago

it should be noted next-mf is not freely available - it is a paid plugin !

ecwyne commented 3 years ago

🤔 @jonniedarko https://www.npmjs.com/package/@module-federation/nextjs-mf

jonniedarko commented 3 years ago

@ecwyne https://github.com/module-federation/nextjs-mf is private repo now and https://twitter.com/ScriptedAlchemy/status/1418508699891425283 from Zack Jackson @ScriptedAlchemy

Who wants to test the updated nextjs-MF plugin? It’s going to be a paid package but am looking for a few beta testers who will get temporary free access while it’s being tested. DM me

ScriptedAlchemy commented 3 years ago

Yep it's no longer a free plugin sorry fam but time is money 💰 v3 is on a paid registry, v2 is the last open release but doesn't work with newer versions of nextjs

Next keeps breaking our support and keeping the project working for everyone requires non free engineering resources.

All / many MF works moving forward will likely cater to paid packages as the OSS R&D is becoming too expensive.

Times be changing. OSS was good while it lasted but funding is just too low.

Can't invest $50k into a package just for the greater good anymore. Losses have to be recouped

On a side note, funded research has allowed us to crack federated SSR in next.js. For those interested we do have a solid solution, so the model is advancing the ecosystem - despite it being an unpopular opinion.

Either we stop coding all together or we find a way to fund our research 🔬

besh commented 3 years ago

@ScriptedAlchemy Thanks for your efforts and understand entirely

v2 is the last open release but doesn't work with newer versions of nextjs

Do you happen to know the last good combination of next and nextjs-mf? Are you saying that all > v11 next is a non-starter? I'm on next 11.0.1 and @module-federation/nextjs-mf and 2.3.1 and noticing two things:

the vendors-node_modules_* files that get generated in the chucks folder of the remote app , (e.g. http://localhost:3001/_next/static/chunks/vendors-node_modules_react_index_js.js) 404 in the consuming app because the consuming app is searching for them on its host (http://localhost:3000/_next/static/chunks/vendors-node_modules_react_index_js.js). I patched this locally with a copy script, but clearly, I'm doing something wrong.
Regardless of my shared settings for react and react-dom, I'm facing invalid hook calls.

I'm not necessarily seeking help on fixing those issues specifically, more so wondering if I'm barking up the wrong tree by attempting to use next 11.

TychosDrunkElk commented 3 years ago

@besh I was able to get next 11 and nextjs-mf 2.3.1 working with a few tweaks (private project, so i cant share but will try to put up an example in a bit):

make sure you are setting config.output.publicPath = 'auto' in your remote app's webpack config
in the consuming app at the top of your _app file, add:
```
if (process.browser) {
window.remoteApp.init(__webpack_share_scopes__.default);
}
```
(see https://webpack.js.org/concepts/module-federation/#dynamic-remote-containers)
make sure react is shared in both the consuming and remote app, with the consuming app having eager: true in addition to requiredVersion: false and singleton: true.

Obviously this wont work for ssr though, excited to see progress made in that area!

EDIT: super simple example w/ a button to demo hooks: https://github.com/TychosDrunkElk/mf-example and https://github.com/TychosDrunkElk/mf-example-remote

deployed to https://mf-example.vercel.app/

TychosDrunkElk commented 3 years ago

OH also I had to use window.remoteApp.get instead of dynamic imports, probably for the same reason described in https://app.privjs.com/buy/packageDetail?pkg=@module-federation/nextjs-mf.

besh commented 3 years ago

@TychosDrunkElk nailed it. config.output.publicPath = 'auto' resolved the chunk issue andwindow.remoteApp.init(__webpack_share_scopes__.default); resolved the sharing issue. I personally did not have to use window.{the_remove_app}.get in my case, but will keep an eye out for issues there.

Thank you for sharing.

ScriptedAlchemy commented 3 years ago

yeah if you use a federated import in many dependency chains attached to index.js, it breaks the build - you have to use the low-level API

ScriptedAlchemy commented 2 years ago

Set me back a few months but I am able to use it without much restriction. Even got HOT prod server reloading & SSR to work.

Still. Would appreciate a damn async boundary around the entry point ffs.

At least buildHttp is optional now, still doesn't help at runtime stuff.

Sucks for us as users tho, but hey at least it's not like it was with Next 9 which was close to unusable with MF

Still, 3 major versions and no movement at all? They have the authors of two major compilers on staff as well & we aren't exactly talking rocket science here to get it enabled internally...

lu-zen commented 2 years ago

@ScriptedAlchemy did you ever explored if it's possible to use MF with Remix?

ScriptedAlchemy commented 2 years ago

@ScriptedAlchemy did you ever explored if it's possible to use MF with Remix?

Yeah it is. Jacob used to be on my platform team at lulu. Hems at remixx now. He knows MF better than anyone else outside of webpack, together we got MF working on next and in the process he prototyped it on remixx

The trick is you pretty much have a sidecar build (same as next) that's webpack based. So the remote entry and exposed modules go thru webpack. Then you can hook share scope up to ESM modules from esbuild.

Process is near the same for how we make it work in nextjs. Sometime in December we were planning to try and make something out of it.

Spoke with him about it last week actually so this is something we are actively thinking about

yumauri commented 2 years ago

So... Is it currently possible to use Next 12 with Module Federation? Couldn't find any repos with examples, only with previous version.

victordidenko commented 2 years ago

@ScriptedAlchemy does paid version of @module-federation/nextjs-mf supports Next 12 yet?

ScriptedAlchemy commented 2 years ago

Last I tested it works just fine. Nothing next can change impacts my plugin. Federated modules are compiled independently of nexts actual build.

I updated my largest repo today and federation required no change. Mostly Babel problems on the main build lol

ScriptedAlchemy commented 2 years ago

Total speculation. But if vercel plans to support federation specifically it probably will happen when react 18 is released.

I do have federated SSR working already (for like 6 months now)

But R18 would simplify it a little and full async support would be prime for MF support

sirajtechy commented 2 years ago

This Feature is the game changer and the Next js team still thinking to prioritise it. Development becoming costlier these days and Time is becoming so essential to build out a ROBUST, RESILIENT, RELIABLE product. When @ScriptedAlchemy like people are shedding time and effort to bring out the crazy feature up and running for the FE community to leverage it why not WE Embrace it. !! On a side note I am not commercializing Zack here for his effort but truly respect his work to solve the daily problems we engineers face during the development and deployment process :-(

ScriptedAlchemy commented 2 years ago

Apply enough pressure, and vercel will bend to support it. I imagine it's on the roadmap, but if this git issue got a few hundred comments - it would be harder to ignore.

So if you want it sooner and don't want to depend on my commercial packages, get as many developers as possible to comment on this issue.

Strength in numbers and the squeaky wheel gets the grease. We all want it but we need to come together and apply a lot more pressure to make this a top priority item.

I'm not a fan of grayhat tactics. But if you all want this capability sooner - you need to broadcast and get as many people as possible to join this conversation.

Vercel has all the resources needed to make this a reality. I've seen what Tobias can do in mere hours. He runs circles around me.

Chances are he could bring first class support to next within a week if there's enough demand from end users. Gut feeling is this is a product issue and the jira ticket keeps getting pushed down the backlog.

Just saying... The founder of webpack is on staff at this company, this isn't rocket science especially to the author of the compiler itself

lednhatkhanh commented 2 years ago

We really really want to have native module federation support in Next.js!

sirajtechy commented 2 years ago

@ScriptedAlchemy I am already buying ur library and sooner for my Org project too :-)

mehmetpekcan commented 2 years ago

Native module federation support would be great and on the other hand I appreciate Zack for all of his efforts. He is doing great things on this way!

marcelovicentegc commented 2 years ago

I have been following this issue for quite some time now and have tried to figure out ways to build NextJS micro-frontend frameworks without Module Federation. I'm considering jumping into making NextJS and Module Federation compatible, but that would mean a lot of time put in since I have no context of how both work internally as of now, but that seems like the unbeatable way of building MFEs with NextJS.

Zack, thanks for all your efforts! Vercel folks, thanks for bringing the best React framework to life. NextJS being compatible with Module Federation will be revolutionary.

noreiller commented 2 years ago

I have made Next.js work with Module Federation thanks to the work of @ScriptedAlchemy and @telenko with node-mf. It supports server and client side but I missed time to contribute back to the community. I hope to be able to do it next week. 🤞

BhaskaranR commented 2 years ago

we are using nextjs for backoffice tools. it will be helpful if you could support module federation out of box.

tundera commented 2 years ago

This is an awesome feature and would only make Next.js even better

SketchLagoon commented 2 years ago

This would be an invaluable tool in the toolbox for nextjs! 🙌 It's the one thing holding me back from integrating nextjs into some of my projects.

ScriptedAlchemy commented 2 years ago

I have made Next.js work with Module Federation thanks to the work of @ScriptedAlchemy and @telenko with node-mf. It supports server and client side but I missed time to contribute back to the community. I hope to be able to do it next week. 🤞

Let’s talk because I’m looking at doing the same thing.

noreiller commented 2 years ago

I have made Next.js work with Module Federation thanks to the work of @ScriptedAlchemy and @telenko with node-mf. It supports server and client side but I missed time to contribute back to the community. I hope to be able to do it next week. 🤞

Let’s talk because I’m looking at doing the same thing.

Yes, I will contact you next week (I'm in vacation right now).

rmunch commented 2 years ago

Would love to see native Module Federation support in NextJS as well 👍

I have made Next.js work with Module Federation thanks to the work of @ScriptedAlchemy and @telenko with node-mf. It supports server and client side but I missed time to contribute back to the community. I hope to be able to do it next week. 🤞

I've also got client and server federation working thanks to this prior art. It was necessary to add async boundaries in certain places since NextJS doesn't have a central boot point, though I was able to figure out a pattern to do this at the page level and to enforce it with a custom ESLint ruleset - happy to share details if it'd be helpful.

ScriptedAlchemy commented 2 years ago

Im glad we can make this work and bridge some gaps. But it’s important to note that us making it “work” is a massive workaround that literally adds 4 webpack builds to the app.

It’s very important vercel implements support for this feature because we are all building the codebase 4 times to make it work right now

acojuli commented 2 years ago

We love to have native module federation support in Nextjs !

lu-zen commented 2 years ago

Would like to see this feature natively supported in next, it makes mfe so easy, many teams will benefit from this.

danantal commented 2 years ago

This is the kind of feature that would make next be one step ahead over competitors.

nodkz commented 2 years ago

Module federation in Next.js will be the next step for Spotify's BFF architecture when become able to develop frontend UI inside Service teams. It's incredibly important in complex data domains when knowledge distribution between developers is very difficult.

I was expecting the ModuleFederation to appear in NextJS since version 11. And I'm still can't understand why this important feature is not implemented yet.

Right now we have dozens of separate NextJs applications and navigation between them enforces full page reload. And it's a bad user experience for our users but it is less evil according to the complexity of our data domains.

Engineering consists of trade-offs. And we are building our systems from trade-offs. I hope and believe that someday MF will be available in NextJS out of the box.

ScriptedAlchemy commented 2 years ago

Update: I just got Federation working for both client and server via code streaming. I should be ready to release a new plugin in the next week or so.

Still this is a workaround solution for federation on next, but at least federated code will work beyond the client-side. This won't utilize our new federated multiplexing so you need rolling restarts to re-require new federated module updates. But once we have multiplexing finished, one would be able to securely stream any software. Multiplexing will bring us to a state of zero-install, always on, always warm servers that never require a redeploy. It should work with next.js as well since the multiplexing plugin will be altering how webpack loads and runs modules, but multiplexing is designed to stream any language from any location to any machine on demand. (Tvs, drones, browsers, lambdas, workers, standard Docker containers)

First stop is federated Cloudflare workers, and yes, you'd be able to deploy a standard next.js app to a Cloudflare worker. Not the SSG or Cloudflare Pages half-measure but full next.js apps as they exist today running on the edge with no backing Lambda or Infrastructure whatsoever.

Handles multiple files, dynamic imports, code streams, and can execute up to 25mb chunks (can execute multiple chunks)

Initial testing shows "cold start" times of 0.5ms and average uncached execution duration for a SSR app is around 25ms :)

Marcelx8 commented 2 years ago

We would love to use Module Federation with Nextjs SSR to help with our Micro Frontend approach. Nextjs can definitely take lead in this feature.

noreiller commented 2 years ago

node-mf is now supporting Next.js for the server-side part. Now, we are able to consume a getServerSideProps function from another app.

The client-side part (including hydration) needs more work but it's a first step.

ericvel commented 2 years ago

I am currently working on a big web application, where multiple teams will develop separate features that will all be part of the same shared container (header, footer etc.). Using module federation to achieve this seems like the natural way to go, and having it natively supported by Next.js would make our lives infintely easier.

Please don't ignore this issue, Vercel!

alexUXUI commented 2 years ago

As an architect at large enterprises, I work with systems at scale which often require distributing code. Module federation perfectly enables the ability to distribute code and I genuinely think Next.js will capture more of the enterprise market with MF support. Let’s make Next.js the react meta framework for enterprise level needs. This could be a massive point of differentiation from solutions like Remix. Thanks a bunch for your consideration.

vercel / next.js

Module Federation Support #16368

Feature request

Is your feature request related to a problem? Please describe.

Describe the solution you'd like

Current workaround