loadable not working in SSR with webpack module federation -

ajayjaggi commented 3 years ago

🐛 Bug Report

loadable-components: failed to synchronously load component, which expected to be available { fileName: './src/shared/dedicated/index.js', chunkName: 'Dedicated', error: 'Cannot read property \'call\' of undefined' } (node:7562) UnhandledPromiseRejectionWarning: TypeError: Cannot read property 'call' of undefined at webpack_require (/Users/ajay/Desktop/WebStrom/MicroFrontend/Edge/dist/server/main.js:394:42) at Module../src/shared/components/Footer/index.js (/Users/ajay/Desktop/WebStrom/MicroFrontend/Edge/dist/server/Dedicated.server.js:21:71) at webpack_require (/Users/ajay/Desktop/WebStrom/MicroFrontend/Edge/dist/server/main.js:394:42) at Module../src/shared/dedicated/index.js (/Users/ajay/Desktop/WebStrom/MicroFrontend/Edge/dist/server/Dedicated.server.js:60:76) at __webpack_require__ (/Users/ajay/Desktop/WebStrom/MicroFrontend/Edge/dist/server/main.js:394:42) at Object.requireSync (/Users/ajay/Desktop/WebStrom/MicroFrontend/Edge/dist/server/src_server_render_js.server.js:225:14) at InnerLoadable.loadSync (/Users/ajay/Desktop/WebStrom/MicroFrontend/Edge/dist/server/vendors-node_modules_loadable_server_lib_index_js-node_modules_express_index_js-nodemodules-ee7ccd.js:420:35) at new InnerLoadable (/Users/ajay/Desktop/WebStrom/MicroFrontend/Edge/dist/server/vendors-node_modules_loadable_server_lib_index_js-node_modules_express_index_js-nodemodules-ee7ccd.js:315:17) at processChild (/Users/ajay/Desktop/WebStrom/MicroFrontend/Edge/dist/server/vendors-node_modules_loadable_server_lib_index_js-node_modules_express_index_js-nodemodules-ee7ccd.js:56603:14) at resolve (/Users/ajay/Desktop/WebStrom/MicroFrontend/Edge/dist/server/vendors-node_modules_loadable_server_lib_index_js-node_modules_express_index_js-nodemodules-ee7ccd.js:56568:5) (node:7562) UnhandledPromiseRejectionWarning: Unhandled promise rejection. This error originated either by throwing inside of an async function without a catch block, or by rejecting a promise which was not handled with .catch(). (rejection id: 1) (node:7562) [DEP0018] DeprecationWarning: Unhandled promise rejections are deprecated. In the future, promise rejections that are not handled will terminate the Node.js process with a non-zero exit code.

To Reproduce

routes.js -

import React from 'react' import loadable from '@loadable/component' // import Home from './home' // import Dedicated from './dedicated'

const Home = loadable(() => import(/ webpackChunkName: "Home" / './home')) const Dedicated = loadable( () => import(/ webpackChunkName: "Dedicated" / './dedicated'))

const homeRoute = (path) => ({ path, exact: true, component: Home })

const dedicatedRoute = (path) => ({ path, exact: true, component: Dedicated })

export default () => [ homeRoute('/'), dedicatedRoute('/:player(messi)') ]

Expected behavior

A clear and concise description of what you expected to happen.

Link to repl or repo (highly encouraged)

https://github.com/ajayjaggi/MicroForntEnd-Basic-Structure

Issues without a reproduction link are likely to stall.

open-collective-bot[bot] commented 3 years ago

Hey @ajayjaggi :wave:, Thank you for opening an issue. We'll get back to you as soon as we can. Please, consider supporting us on Open Collective. We give a special attention to issues opened by backers. If you use Loadable at work, you can also ask your company to sponsor us :heart:.

ajayjaggi commented 3 years ago

Hi. i saw this PR https://github.com/gregberge/loadable-components/pull/638/files and used these fixes to run loadable in my local along with webpack 5. But further in SSR, wrapping components with loadable breaks the flow...Error is published above. Also i have mentioned my repo link..

StephaneRob commented 3 years ago

Hello @ajayjaggi,

did you try to use correct key for chunkExtractor.js? this https://github.com/gregberge/loadable-components/pull/638/files#diff-ae8b2f8a1736c52b65efbc3c341977cb11ce566027e0befaf874569fe969dcadR208 instead of https://github.com/ajayjaggi/MicroForntEnd-Basic-Structure/blob/master/Edge/node_modules/%40loadable/server/lib/ChunkExtractor.js#L208

example

return {
-     cleanFilename,
+     filename: resolvedFilename,
...

ajayjaggi commented 3 years ago

Thanks for this.I used the wrong key. Have corrected this but still getting the same error.

StephaneRob commented 3 years ago

Did you use webpack LoadablePlugin in your webpack config?

ajayjaggi commented 3 years ago

yes for both the websites ie Edge and Demand i have used LoadablePlugin in the webpack config for client side. demand website - https://github.com/ajayjaggi/MicroForntEnd-BasicStructure/blob/master/Demand/config/webpack/client.js edge website - https://github.com/ajayjaggi/MicroForntEnd-Basic-Structure/blob/master/Edge/config/webpack/client.js

ajayjaggi commented 3 years ago

https://github.com/ajayjaggi/MicroForntEnd-Basic-Structure - this is the PR for project. The error of 'Cannot read property 'call' of undefined' is occurring only in server side rendering.For client side everything is working fine

theKashey commented 3 years ago

It may take a while before it will start working properly, but 🤞

ajayjaggi commented 3 years ago

Hi @StephaneRob / @theKashey ,can we connect over a call to discuss the issue ? We can try to find the solution together.

theKashey commented 3 years ago

@ScriptedAlchemy is the boss in MF lands. We all need his help, not my help. PS: https://github.com/gregberge/loadable-components/issues/635 sounds to be related. We need unique callbacks for every remote. Or we don't?

ScriptedAlchemy commented 3 years ago

Each host has a name. Webpack usually appends a name to the remote containers. So they get their own name.

Webpack accesses the runtime via window.app1.get. App1 is the name you call it in the config.

I'm not sure if jsonp matters because they all get namespaced. It would have to be tested but I use loadable and do not have problems. However I'm using MF not standard entry-point runtimes which execute differently.

ScriptedAlchemy commented 3 years ago

Quoting from a comment I left on recently merged PR:

If you have the remotes on the page immediately. Like hardcoded. All remote code should transport down in a single RTT.

If you want to SSR the chunks a remote requires:

loadable needs a new way to map names of remotes to federated chunk maps. I need to extend the MF api so remotes will export their chunk maps so loadable can extract them during render. Whenever you use loadable. Babel basically transforms the import into a object with a normal require statement and the module / chunk id. Then it's executed the HOC pushes these IDs into a scope that the server can access after the react render.

The only Id it will have is ./someExposedModule - and we would need to go over the remotes stats and find what the browser side script is for that module.

Same problem exists in next js. The loadable method straight crashes. No existing code split tool is capable of MF chunk coronations. I work around hydration issues with top level awaits. Which work on both client and server.

You could also use partial hydration and simply hydrate markup when visible.

I don't foresee this being hard to accomplish. Especially since I can extend MF apis at will.

We could even change the getter on the remote so whenever a server gets code from the remote, it's container will report what request was asked for from the remote. It actually could be done without and need for Babel at all, or even loadable HOC. The only part we would need to adapt is ChunkExtractor to read from the second scope the remote containers are accessing directly.

This mechanism would work exactly like react-universal does. Push right into a map, but we will be doing this inside the webpack runtime directly.

MF should not be a complex problem to render correctly on. You don't need to use loadable as a wrapper 👆 we only need to flush chunks into it from webpack runtime.

ScriptedAlchemy commented 3 years ago

@theKashey how can I import and push extra chunk names into your context.

How can we have loadable read the remote containers chunks. I can have webpack infer this from the plugin.

What we could do is provide a loadableFederationPlugin which passes the object to MF but we can read the info.

Then we would have the paths to all remotes and can require the chunk stats upfront.

I could write to a map you expose directly in webpack. As the runtime attaches, it sends context to the host. The "hard" part on the webpack side is emitting the stats and reducing it down to simple maps.

If I'm able to push chunk maps and the module as it's required at runtime. This would make it much simpler. Then I wouldn't need to provide stats upfront but could push them into loadable as the remote attaches on its own.

ScriptedAlchemy commented 3 years ago

Okay, I'm on my iPad to cant really code much. Here's what the startup code will look like. (Check module federation examples/startup-code) to see it configured in webpack.

This entrypoint would be part of webpack-imported, and its added to webpack with something like SingleEntryPlugin or EntryPlugin (or mutation of options, like u do in next) We can get app1 from webpack and use template strings, or we can compose variables via definePlugin, like you do to env vars.

Global.chunkMap = Map() // or whatever
const container = __webpack_require__('webpack/container/entry/app1');
// app1 could be process.env.CURRENT_HOST or something we can inject during build. 
Module.exports = {
  get(request) {Map.add(request); return container.get(request)}
  init() {return container.init(arguments)}
  ClientChunks: non_webpack_require('../clientStats.json') // or some injection to inline this after the fact. 
}

We can also use normal require and stuff in here, its just a entrypoint.

As i use federated import(scope/request), RemoteModule will perform global.[scope].get(request)

Since we own the getter, its going to push into a global map. We can then read config from MF plugin to get paths to remote container if we need.

ImportedFederatedPlugin({ Name:app1 Remotes: {otherRemote: path.resolve(pathToRemote)} })

Now you've got the options passed to MF. So we can infer name, and know how to find the container to get the exported client chunks off it.

You can also use, in the server. So chunkExtractor would go:

remoteChunks = __webpack_require__(require.resolve(remote)).ClientChunks

Alternatively, we could attach to scope during initialization

{init()=>{Object.assign(global.chunkMaps,ClientChunks)}}

Inside loadable, you'd pretty much want to prevent the typical babel transform. Instead we use webpack directly. We can use require if needed for server transform, then webpack will hoist to app boot time, i think.

Lastly, we could use the existing HOC to push ones actually rendered, depending on if the require function pull the chunks right away. Ideally if we can preloadAll or something it might be better.

There's likely some details to work out but this will work.

ajayjaggi commented 3 years ago

hi @ScriptedAlchemy, i was very curious about finding the solution to the problem. If we could talk and you could guide me through, i would be very happy to contribute.

ScriptedAlchemy commented 3 years ago

There's other issues that block webpack support besides this :/

But I'll push a branch somewhere soon that you can look at

ajayjaggi commented 3 years ago

Hi @ScriptedAlchemy , If you could share the branch it would be great help

ScriptedAlchemy commented 3 years ago

I'm building most of this capability into the module federation dashboard.

I'll backport so capabilities to standalone

stale[bot] commented 3 years ago

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

visormatt commented 3 years ago

~@theKashey Can we re-open this guy as~ it's a blocker for users of this package from updating to Webpack 5? Also is there any idea of timing or any assistance we "the community" can help with? CC. @ScriptedAlchemy

theKashey commented 3 years ago

webpack 5 support was already PR-ed - #676, I am waiting for a little free time to get it merged.

visormatt commented 3 years ago

Amazing, I should have poked around the branches a bit more, I didn't see it linked here 🤷. Thanks again and thank you for all the fantastic work thats gone into this project!!

AlexSergey commented 3 years ago

webpack 5 support was already PR-ed - #676, I am waiting for a little free time to get it merged.

Any progress?

pauliusuza commented 3 years ago

Any progress on this? Seems to break razzle builds (hash mismatch on production build)

suhanw commented 3 years ago

Watching this issue as well.

I also created a reproduction repo. README has instructions on how to run locally. https://github.com/suhanw/wp5-modfed-ssr-loadable

A very simple setup where remote exposes TopNav component, and host consumes TopNav. https://github.com/suhanw/wp5-modfed-ssr-loadable/blob/main/host/client/components/App/index.js

const TopNav = loadable(() => import('remote/top-nav'), { ssr: true }); The line above throws the following error:

loadable-components: failed to synchronously load component, which expected to be available {
  fileName: 'webpack/container/remote/remote/top-nav',
  chunkName: 'remote-top-nav',
  error: '__webpack_modules__[moduleId] is not a function'
}
(node:62803) UnhandledPromiseRejectionWarning: TypeError: __webpack_modules__[moduleId] is not a function
    at __webpack_require__ (/Users/suhanwijaya/Desktop/temp/wp5-modfed-ssr-loadable/host/build/server/bundle.js:249:41)
    at Object.requireSync (/Users/suhanwijaya/Desktop/temp/wp5-modfed-ssr-loadable/host/build/server/server_renderer_js.bundle.js:60:14)
    at InnerLoadable.loadSync (/Users/suhanwijaya/Desktop/temp/wp5-modfed-ssr-loadable/host/node_modules/@loadable/component/dist/loadable.cjs.js:278:35)
    at new InnerLoadable (/Users/suhanwijaya/Desktop/temp/wp5-modfed-ssr-loadable/host/node_modules/@loadable/component/dist/loadable.cjs.js:173:17)
    at processChild (/Users/suhanwijaya/Desktop/temp/wp5-modfed-ssr-loadable/host/node_modules/react-dom/cjs/react-dom-server.node.development.js:3305:14)
    at resolve (/Users/suhanwijaya/Desktop/temp/wp5-modfed-ssr-loadable/host/node_modules/react-dom/cjs/react-dom-server.node.development.js:3270:5)
    at ReactDOMServerRenderer.render (/Users/suhanwijaya/Desktop/temp/wp5-modfed-ssr-loadable/host/node_modules/react-dom/cjs/react-dom-server.node.development.js:3753:22)
    at ReactDOMServerRenderer.read (/Users/suhanwijaya/Desktop/temp/wp5-modfed-ssr-loadable/host/node_modules/react-dom/cjs/react-dom-server.node.development.js:3690:29)
    at Object.renderToString (/Users/suhanwijaya/Desktop/temp/wp5-modfed-ssr-loadable/host/node_modules/react-dom/cjs/react-dom-server.node.development.js:4298:27)
    at _callee$ (/Users/suhanwijaya/Desktop/temp/wp5-modfed-ssr-loadable/host/build/server/server_renderer_js.bundle.js:134:76)

import TopNav from 'remote/top-nav'; This line above appears to work, but extractor.getStyleTags() returns empty string (this seems related to https://github.com/gregberge/loadable-components/issues/706)

@ajayjaggi curious if you found a workaround?

GitProdEnv commented 3 years ago

Any progress?

I get the same error

loadable-components: failed to synchronously load component, which expected to be available {
  fileName: './src/components/Mfa_1_app.tsx',
  chunkName: 'components-Mfa_1_app',
  error: '__webpack_modules__[moduleId] is not a function'
}
(node:29690) UnhandledPromiseRejectionWarning: TypeError: __webpack_modules__[moduleId] is not a function
    at __webpack_require__ (/media/gitprodenv/Elements6/Coding/mfa/packages/container/dist-backend/webpack:/@microfrontend/container/webpack/bootstrap:19:1)
    at Object../src/components/Mfa_1_app.tsx (/media/gitprodenv/Elements6/Coding/mfa/packages/container/dist-backend/dev-server-bundle.js:3086:73)
....

muehre commented 3 years ago

Same failed to synchronously load component, which expected to be available Any updates?

theKashey commented 3 years ago

There is a little problem with supporting module federation, and it short it's all about relying on a stats.json(loadable.json) to handle SSR case. One loadable, one LoadableWebpackPlugin, one stats, one application - federated pieces will be not visible for the ChunkExtractor and magic will not happen.

Without first class support from Federation, and corresponding changes to LoadablePlugin - it's not gonna work.

7rulnik commented 3 years ago

Also you can check https://github.com/module-federation/module-federation-examples/issues/751#issuecomment-832885498 and https://github.com/jacob-ebey/ssr-react-streaming-example/blob/main/ssr_react_streaming_example_b/federation-stats-plugin.js

theKashey commented 3 years ago

Basically that's all - collect stats from every federated module and merge result together to ChunkExtractor. To something like FederatedChunkExtractor.

Some extra functionality might be required to keep server and client "choices" consistent, as well as to support orchestration, but the very essence of proper MF support - just get all stats together in one place.

stale[bot] commented 3 years ago

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

ScriptedAlchemy commented 3 years ago

We have built a solve for this, however we need some library updates to be performed to pair our technology with this library

theKashey commented 3 years ago

Let's talk about this 🤙

shivam611996 commented 3 years ago

Hello everyone,

I am facing the following issue while loading the web page: loadable-components: failed to synchronously load component, which expected to be available

I also get errors like this:

react-dom.js:141 TypeError: Cannot read property 'call' of undefined at a (app.c859347b3447f4265e89.js:2) at Module.5470 (chunk.Bill.c859347b3447f4265e89.js:1) at a (app.c859347b3447f4265e89.js:2) at Object.requireSync (app.c859347b3447f4265e89.js:2) at n.i.loadSync (app.c859347b3447f4265e89.js:2) at new n (app.c859347b3447f4265e89.js:2) at fh (react-dom.js:81) at Ch (react-dom.js:114) at Pj (react-dom.js:233) at di (react-dom.js:168)

FYI, I am using the following library versions: "webpack": "^5.37.1", "@loadable/component": "^5.14.1"

Could someone help me with this issue, please?

@ScriptedAlchemy @theKashey please let us know in case of any updates.

Thanks!

shivam611996 commented 3 years ago

any updates. please?

wvanrensselaer commented 3 years ago

@ScriptedAlchemy is the solve you mention the same as described in your Next.js/SSR article? In which case is it tied to Next.js use?

DanielAmenou commented 3 years ago

In order to solve this issue on the browser side we need to overcome two problems: 1 - We don't have the chunks that are needed for rendering the components; 2 - loadable-components trying to get the mf chunks by using the following id - webpack/container/remote/appName/moduleName while the remote uses different id;

In order to solve these problems, I created two plugins - "webpack-federation-stats-plugin" and "webpack-federation-module-id-plugin"

webpack-federation-stats-plugin - creates a JSON file that contains all the required chunks of the exposed modules

webpack-federation-module-id-plugin - changes the module id of the exposed modules, so the id will be in the following form: webpack/container/remote/appName/moduleName

Without the plugin:

(self["__LOADABLE_LOADED_CHUNKS__"] = self["__LOADABLE_LOADED_CHUNKS__"] || []).push([["src_Button_js"],{
"src/Button": ((__unused_webpack_module, __webpack_exports__, __webpack_require__) => {...}
}

With the plugin:

(self["__LOADABLE_LOADED_CHUNKS__"] = self["__LOADABLE_LOADED_CHUNKS__"] || []).push([["src_Button_js"],{
"webpack/container/remote/MyAppName/Button": ((__unused_webpack_module, __webpack_exports__, __webpack_require__) => {...}
}

After adding these plugins to the remote you need to get the "federation-stats.json" in the host and inject the required chunks into the Html (you should check which components were rendered and send only the relevant chunks). You can get the rendered components in that way:

const getLoadableRequiredComponents = (extractor) => {
    const loadableElement = extractor.getScriptElements().find((el) => el.key === "__LOADABLE_REQUIRED_CHUNKS___ext")
    const {namedChunks} = JSON.parse(loadableElement.props.dangerouslySetInnerHTML.__html)
    return namedChunks
}

The mf components names are in the following form appName-componentName

nescalante commented 2 years ago

@DanielAmenou do you have a working example for those plugins?

theKashey commented 2 years ago

[ ] @theKashey to work with @ScriptedAlchemy and @DanielAmenou to create documentation around MF and code-splitting anyone can follow

DanielAmenou commented 2 years ago

@nescalante I'm working on it, I'll share it soon.

ScriptedAlchemy commented 2 years ago

has been solved as referenced in the PR to MF-examples. Probably could have tighter integration with the core library, but not a showstopper

theKashey commented 2 years ago

It can be started from the documentation side. I am looking forward to refactor ChunkExtractor a little to make webpack-plugin and getAnyScriptTags interchangeable which will easy integration with MF, or any other technology if needed.

philmoreira commented 2 years ago

Given the above (using "webpack-federation-stats-plugin" and "webpack-federation-module-id-plugin") - is it appropriate to post issues with SSR/loadable/module-federation here?

Should we wait on better documentation? Should we wait on the above mentioned refactor? Any clarity on that would be awesome!

And many thanks to all so far on all the work done to get us to this point :)

fivethreeo commented 2 years ago

Wonder how this would work with react18. May not even need loadble-components when react supports it properly with streaming.

Maybe in razzle 5 😀

ScriptedAlchemy commented 2 years ago

I've long ago solved chunk flushing for code split components. All you need to do is merge the stats json chunk that loadable emits, either as env variable or a api call to static asset before flushing chunks.

React 18 supports streaming so this won't be a problem however you'd still want to flush out the JS upfront to avoid round trips when webpack kicks in.

I do already have react 18 level support for federated SSR, async components, and component level data fetching (had it for about 7 months) - however my pattern depends on discovering side effects during render so I'm not suspending leaf nodes server side. Just traverse the app tree before renderToString and flush out and component level data or federated imports, then store it on react context so when renderToString is called, I've already promise.all'd and have the resolved promises available synchronously.

Traversing the app tree added about 80ms of overhead to my lambdas, a small price to pay for production sites that are cached 90% of the time anyways lol

theKashey commented 2 years ago

With React 18 it is possible to do SSR and later hydration without any other library involved in the process, however:

unless you use "inlining" CSS-in-JS or critical style extraction you need something like Loadable to add used styles in the page header.
you might suffer from render-as-you-fetch, aka waterfall-ish script loading. You will need an extra tool to discover used scripts and flush them from SSR or the outcome can be completely unbearable. This is greatly dependent on the application and especially application size.
@ScriptedAlchemy - it would be nice to have some stats.json compatible plugin to produce a "federated medium" other libraries can use. In the case of loadable there loadable/webpack-plugin is more than replaceable with any other "source". Would it be enough, and changes to which other systems might be required (like https://github.com/gregberge/loadable-components/pull/7010?

ScriptedAlchemy commented 2 years ago

We released a webpack federated stats plugin many months ago to solve chunk flushing at server time. It works just fine under my use cases, haha, but I have custom everything. Someone could send a Pr and adjust the stats emitted, or an adapter output could be added for loadable to perform lookups similar to how it does today

theKashey commented 2 years ago

Ok, I know what you are talking about. I'll take a deeper look

saipatro commented 2 years ago

Hey guys, the PR is still in open state! Any updates here? @ScriptedAlchemy @theKashey

ScriptedAlchemy commented 2 years ago

I’m about to release this flush chunks capability for next.js federated SSR plugin.

Here’s how i did it. During server build, I attach a small chunk map to module federation apis. So when i call the remote container on a server, it as an object of mappings.

I read the loadable context which has a mapping of all dynamic imports, the federated ones will have an empty array since theres no known files. I loop over the original manifest and attach any matching imports to the remote modules onto that object. Now when they are flushed out, loadable also flush’s out the federated modules alongside it. To deal with asset prefix, i created something similar to document.currentScript for node - this lets me convert relative paths to absolute urls inside the consumer on the fly.

gregberge / loadable-components