module-federation / module-federation-examples

Implementation examples of module federation , by the creators of module federation
https://module-federation.io/
MIT License
5.64k stars 1.75k forks source link

How typesafe can a remote be with Typescript? #20

Closed echarles closed 2 years ago

echarles commented 4 years ago

Update from @ScriptedAlchemy - https://www.npmjs.com/package/@module-federation/typescript has been released

I can see a disadvantage using remotes libraries vs the classical libraries where e.g. typescript can be used to enforce strong types. The current typescript example defines a generic app2/Button which is similar defining a any type.

https://github.com/module-federation/module-federation-examples/blob/cf2d070c7bbb0d5f447cfe33f69e07bcf85f8ddb/typescript/app1/src/app2.d.ts#L3-L7

As the remote var does not have type, I don't see a way to benefit from the app2 types in app1. Any idea on this?

echarles commented 4 years ago

@ScriptedAlchemy

saulshanabrook commented 4 years ago

I believe, in our case, we should be able to be fine with not having type checking for remote libraries. We can use shared libraries for extension authors import core libraries, so they don't need to be remote and dynamically imported.

echarles commented 4 years ago

I have tried a hacky way to benefit from type-safety importing the definition of the remote project (app2, here widgets) in the host project (app1). As shown on the below image, lazy returns the correct type and non acceptable props are underlined in red. I guess a solution would be to publish the types separately and a 3rd party could depend on those types.

Screenshot 2020-05-05 at 17 51 48
ScriptedAlchemy commented 4 years ago

Or have a monorepo of types

maraisr commented 4 years ago

What I have done is use node to generate a .d.ts file, that are loaded by our global.d.ts file.

So each Remote outputs a .d.ts file that go declare module "dlaWidgets/Widets" {}, with using the ts.createProgram you can tap into the typeChecker stuff that i use to print its public api. Kinda like getting tsc to ouput definition files, but instead spit them into a single file.

Then each of my hosts import this .d.ts file, which I have registered in a global.d.ts, that all of my projectReferenced monorepo apps include.

So there is a small disconnect between a webpack config, and the types. But any exposed things, are automagically available with types in my entire app. So ts might say its a valid import, but just need to actially configure it with webpack also.

Working on open sourcing this generative thing - just gotta make sure it works in all use cases.

echarles commented 4 years ago

@maraisr your approach sounds very interesting. Keep us posted if you open source it.

ScriptedAlchemy commented 4 years ago

ill raise this with Microsoft - they will have to support federation since they are going to be using it heavily

echarles commented 4 years ago

ill raise this with Microsoft - they will have to support federation since they are going to be using it heavily

Awesome!

zhangwilling commented 4 years ago

mono repo is not recomended when our app is very large,independent repo is better.

💡We can watch and output independent repo's d.ts, then start a server which can make d.ts could be downloaded. When other apps need d.ts,just download it to your project from local server.

ckken commented 4 years ago

same issue and suggest like deno remote url type.d.ts

ScriptedAlchemy commented 4 years ago

I found some devs solving this by modifying TS loader. Specifically for module Federation

ckken commented 4 years ago

how to fix it

zhangwilling commented 4 years ago

i think it should be solved by IDE, such as vscode.

ScriptedAlchemy commented 4 years ago

yeah IDE can solve it, assuming it supports that. Meeting with Microsoft in august - seeking changes to TS to support MF, in doing so IDEs will have to follow the spec

ScriptedAlchemy commented 4 years ago

This remains a solution

What I have done is use node to generate a .d.ts file, that are loaded by our global.d.ts file.

So each Remote outputs a .d.ts file that go declare module "dlaWidgets/Widets" {}, with using the ts.createProgram you can tap into the typeChecker stuff that i use to print its public api. Kinda like getting tsc to ouput definition files, but instead spit them into a single file.

Then each of my hosts import this .d.ts file, which I have registered in a global.d.ts, that all of my projectReferenced monorepo apps include.

So there is a small disconnect between a webpack config, and the types. But any exposed things, are automagically available with types in my entire app. So ts might say its a valid import, but just need to actially configure it with webpack also.

Working on open sourcing this generative thing - just gotta make sure it works in all use cases.

glebmachine commented 4 years ago

@zhangwilling it's not a part of IDE, it's all about typechenking of typescript compiler itself

maraisr commented 4 years ago

I like the IDE idea @zhangwilling, but that will be more about aiding the development flow. However... I think this should still remain in the compiler itself, or a definition file that the compiler reads in. Personally feel its the latter.

If you solve to get the compiler to be aware of the remotes, then you don't have to worry about IDE, and specific IDE support as all IDE's will get this knowledge for free.

My solution outlined above still holds true. Its not terribly bad to implement. I'm hoping to get something open sourced soon.

Just wanting to get an idea from the community:

  1. Are you using a monorepo?
  2. Are your exposes objects exposed as standard javascript/typescript files? (Rather than inline in your webpack.config.js)
  3. Are the request's standard node-esk requests, ie no @svgr/webpack!./icon.svg, where the request would firsts require a loader?

I've taken some assumptions to how I build software, but just wanting to get more scope so that this tool when released would solve issues for the majority.

zhangwilling commented 4 years ago

@zhangwilling it's not a part of IDE, it's all about typechenking of typescript compiler itself

@glebmachine yes, it is not part of IDE. Maybe TS can provide extra typing file config, but IDE plugin can do this, it could be improved much by IDE. Same as maven in IDEA, maven is not part of IDEA, but it does. Because it's a very important infrastrure.👻

zhangwilling commented 4 years ago

I like the IDE idea @zhangwilling, but that will be more about aiding the development flow. However... I think this should still remain in the compiler itself, or a definition file that the compiler reads in. Personally feel its the latter.

If you solve to get the compiler to be aware of the remotes, then you don't have to worry about IDE, and specific IDE support as all IDE's will get this knowledge for free.

My solution outlined above still holds true. Its not terribly bad to implement. I'm hoping to get something open sourced soon.

Just wanting to get an idea from the community:

  1. Are you using a monorepo?
  2. Are your exposes objects exposed as standard javascript/typescript files? (Rather than inline in your webpack.config.js)
  3. Are the request's standard node-esk requests, ie no @svgr/webpack!./icon.svg, where the request would firsts require a loader?

I've taken some assumptions to how I build software, but just wanting to get more scope so that this tool when released would solve issues for the majority.

@maraisr monorepo is not recommend when facing large project. so a large project would be seprated to many litttle repos by domain or module. But, we usually would put them together into same parent directory or we use worksapce in VSCode to oragnize them together logically 😈.

glebmachine commented 4 years ago

@zhangwilling yeah

But, typescript are trying to resolve imports. And fails when facing federation module import declaration. It looks like we have to be able to declare special federation modules right into tsconfig.json file or similar way.

For now, i'm workaround this with by loads file by document.createElement('script').

micmro commented 4 years ago

Depending on your use case/setup you could solve this by having a tsconfig.json in the project root and leverage path-mapping to resolve the apps entry points (Remote Name + Exposed module):

/tsconfig.json:

...
"paths": {
      "app1/Remote": ["./packages/app1/src/app"],
      "app2/Remote": ["./packages/app2/src/app"]
}
...

/packages/app1/tsconfig.json & /packages/app2/tsconfig.json:

{
  "extends": "../../tsconfig.json"
}

I've setup a little prototype using this setup: https://github.com/rangle/federated-modules-typescript

maraisr commented 4 years ago

You got to be careful with that approach though. It does mean that you cannot also use path mappings for other things in your app, because if you are you're probably also using tsconfig paths webpack resolver, in which case module federation plugin can't "remote" those. You can but you'd have to have a special build-time tsconfig that excludes the MF mappings.

micmro commented 4 years ago

Thanks @maraisr , that is a very good point I had not considered. I suppose then it is a trade-off of manual remote interface vs potential manual MF path-mapping overwrites.

doerme commented 4 years ago

1.webpack provide a template for the "d.ts" module building 2.webpack config decare the use of remote "d.ts" module 3.IDE support using static inspection for the second point

ckken commented 4 years ago

fix it with npm-dts create in dist & request by app

mizx commented 4 years ago

What I have done is use node to generate a .d.ts file, that are loaded by our global.d.ts file.

So each Remote outputs a .d.ts file that go declare module "dlaWidgets/Widets" {}, with using the ts.createProgram you can tap into the typeChecker stuff that i use to print its public api. Kinda like getting tsc to ouput definition files, but instead spit them into a single file.

Then each of my hosts import this .d.ts file, which I have registered in a global.d.ts, that all of my projectReferenced monorepo apps include.

So there is a small disconnect between a webpack config, and the types. But any exposed things, are automagically available with types in my entire app. So ts might say its a valid import, but just need to actially configure it with webpack also.

Working on open sourcing this generative thing - just gotta make sure it works in all use cases.

In addition to this approach, what about packaging that d.ts ouput file and deploying it as a tar.gz alongside your webpack artifacts? Similar to a @types/* package except not published to registry. Then a host app can optionally add dev-dependency referencing the remote tar.gz file and always have latest TS types.

I think this approach would work well for non-monorepo cases.

gthmb commented 4 years ago

I would love some feedback on how I am currently solving this issue:

I have a mono-repo with several packages. Each package defines a federation.config.json file that declares its remote name and what it exposes, like so:

packages/app2/config/federation.config.json

{
    "name": "app2",
    "filename": "remoteEntry.js",
    "exposes": {
        "./Button": "./app2/Button"
    }
}

The webpack config file for the module imports this config and spreads it into the plugin config like:

packages/app2/config/webpack.config.js

plugins: [
    new ModuleFederationPlugin({
        ...federationConfig,
        shared: {
            ...deps,
        },
    }),
],

I then have an NPM module I am calling federated_types which exposes a CLI command called make-federated-types which can be called via an npm script in a package.

This command finds and reads the package's federation.config.json and uses the TS compiler to generate types with a similar process to what @maraisr described - however I write the types into an @types/__federated_types directory under the projects node_modules directory. This is nice because there is no extra resolution configuration needed in the tsconfig file or webpack config files since they look in @types as a default location.

The command does allow for specifying where the federated typing files should be written, but by default, it writes to the node_modules/@types/__federated_types directory.

Then in each package.json, I added an entry to the scripts node, like:

  "scripts": {
        "make-types": "make-federated-types"
    },

And since I'm using Lerna, when I run lerna run make-types it generates the typings for the items I have exposed via federation

Does this seem like a sound approach? Is writing to the node_modules directory a bad idea? Is there a better location to write to?

ScriptedAlchemy commented 4 years ago

As long as node resolves properly to the location you write to, it's a sound approach. I think haha.

What one could do is create a GitHub crawler that pulls other repos TS def files remotely (similar to webpack code streaming) and write them on demand. You'd only need a list of repos / remotes to "look" for JS defs to pull ahead of time.

Basically your mono repo idea. Applied to polyrepo and using the network

If anyone wants to try building this- I'll give you maintainer access to a repo under the official MF organization

gthmb commented 4 years ago

Thanks for the feedback!

For your suggestions around a poly-repo solution, are you thinking an application would have a config file that defined the repos/remotes they are using, and then a script would pull those repos and generate local copies of the types needed?

Just trying to get a clearer picture. I would be happy to work on something like that. Happy and eager to help how I can!

ScriptedAlchemy commented 4 years ago

Yeah basically. Imagine you listed an array of objects that contain info about your remotes.

Then we could search git repo via api or just the TS defs related to remote files and write those strings to disk in your types directory

Basically replaced fs with fetch and we get the raw git text from api

ScriptedAlchemy commented 4 years ago

This is easier to instrument with federation dashboard because it know what hosts uses what remotes and where they are. In the future I could tack something into dashboard so one could query dashboard for the defs - then write the strings.

gthmb commented 4 years ago

Very cool. I am happy to help!

If it's useful to anyone looking at sharing type definitions between their packages in a mono-repo, here is what I am using: https://github.com/pixability/federated-types

ScriptedAlchemy commented 4 years ago

Will check this out and see what's needed to make this work for both poly and monorepo

flyyuan commented 4 years ago

Here is a better solution with Typescript. Form a closed loop of types, from generation to reference complete for Module Federation project. https://github.com/efoxTeam/emp/tree/main/packages/emp-tune-dts-plugin

gthmb commented 3 years ago

Cool, I'll check that out too. Thanks!

rickihastings commented 3 years ago

Has anyone got a reliable solution for projects that don't use a monorepo?

ckken commented 3 years ago

Has anyone got a reliable solution for projects that don't use a monorepo?

https://github.com/efoxTeam/emp/tree/main/packages/emp-tune-dts-plugin

lm2almeida commented 3 years ago

Is anyone having issues with eslint when importing a federated module?

image image

:point_down: eslint config

{
  "env": {
    "browser": true,
    "jest": true
  },
  "parser": "@typescript-eslint/parser",
  "parserOptions": {
    "ecmaFeatures": {
      "jsx": true
    },
    "ecmaVersion": 12,
    "sourceType": "module"
  },
  "extends": [
    "airbnb",
    "plugin:react/recommended",
    "plugin:@typescript-eslint/eslint-recommended",
    "plugin:@typescript-eslint/recommended",
    "plugin:jest/recommended",
    "prettier/@typescript-eslint",
    "plugin:prettier/recommended"
  ],
  "plugins": ["react", "react-hooks", "@typescript-eslint", "jest", "prettier"],
  "rules": {
    "import/extensions": "off",
    "import/no-extraneous-dependencies": ["error", { "devDependencies": true }],
    "react-hooks/rules-of-hooks": "error",
    "react-hooks/exhaustive-deps": "warn",
    "react/prop-types": "off",
    "react/react-in-jsx-scope": "off",
    "react/jsx-uses-react": "off",
    "react/jsx-filename-extension": [1, { "extensions": [".ts", ".tsx"] }],
    "react/jsx-one-expression-per-line": "off",
    "import/prefer-default-export": "off",
    "@typescript-eslint/no-unused-vars": "error",
    "@typescript-eslint/explicit-module-boundary-types": "off",
    "prettier/prettier": "error"
  },
  "settings": {
    "import/resolver": {
      "typescript": {}
    }
  }
}

:point_down: tsconfig

{
  "compilerOptions": {
    "allowJs": true,
    "allowSyntheticDefaultImports": true,
    "esModuleInterop": true,
    "forceConsistentCasingInFileNames": true,
    "isolatedModules": true,
    "jsx": "react-jsx",
    "lib": ["dom", "dom.iterable", "esnext"],
    "moduleResolution": "node",
    "noEmit": true,
    "noImplicitAny": true,
    "noImplicitReturns": true,
    "resolveJsonModule": true,
    "skipLibCheck": true,
    "strict": true,
    "baseUrl": "./src",
    "paths": {
      "routes/*": ["routes/*"],
      "services/*": ["services/*"],
      "pages/*": ["pages/*"],
      "components/*": ["components/*"]
    }
  },
  "include": ["src"]
}

Thanks! :pray:

rickihastings commented 3 years ago

@lm2almeida I think the issue is that your TypeScript project doesn't understand what remote is, and therefore eslint doesn't understand it as your eslint is configured to use TypeScript as the import resolver.

As for a solution to the TypeScript issue. The only solutions I've found so far are in this thread and they involve building the types separately and then importing them from somewhere, the main issues I'm seeing with this are the manual steps of building the types, and then syncing up these types across your projects when the files change.

At the moment it's an unsolved problem imo. It's less of an issue in a monorepo though.

lm2almeida commented 3 years ago

@lm2almeida I think the issue is that your TypeScript project doesn't understand what remote is, and therefore eslint doesn't understand it as your eslint is configured to use TypeScript as the import resolver.

As for a solution to the TypeScript issue. The only solutions I've found so far are in this thread and they involve building the types separately and then importing them from somewhere, the main issues I'm seeing with this are the manual steps of building the types, and then syncing up these types across your projects when the files change.

At the moment it's an unsolved problem imo. It's less of an issue in a monorepo though.

Thanks for the answer, @rickihastings. The problem here is that even with the declared typing the error still occurs. Looking forward for a solution :pray:

image

Kjaer commented 3 years ago

Whoever stuck on typings here is my approach.

First of all, I am relying on @gthmb 's federated-types package: https://github.com/module-federation/module-federation-examples/issues/20#issuecomment-720719885

on my remote app, I created federation-config.json

/* federation.config.json */

{
  "name": "blank",
  "exposes": {
    "./RemoteApp": "./src/bootstrap"
  }
}

then, I exporting my typings in a custom @types directory on my root. (remote app)

scripts: {
    "make-types": "make-federated-types --outputDir ./@types/remote-app"
}

federated-types export types inside node_modules as default behaviour. For that reason it does not create package.json if custom output directory set. However, I need the package.json because I am adding my remote app's type as a dependency to my host application.

after running make-types command, it generates the types by looking at my federation-config.json and I manually created package.json. In the end, you should have something similar:

├── @types
│   ├── remote-app
│   │   ├── index.d.ts
│   │   ├── remote-app.d.ts
│   │   ├── package.json

I have my typings ready to be import as a dependency for my host app. On my host app's package.json I added these typings as devDependency

devDependencies: {
 "@types/remote-app": "file:../federated-apps/remote-app/@types/remote-app",
}

then run the yarn install as usual, and all my remote-app's typings are available for the host app, without a hassle. What's good at this, all these typings creations and importing to host app is automated and less error-prone.

grzegorzjudas commented 3 years ago

I was able to get my project with module federation and typescript working, along with typesafety of remote components, with minimal cost:

in this scenario the lazy-loaded components are still reporting missing/incorrect props passed. The only con I can see here is that you need to make sure every newly exposed component is added to the .d.ts file, but that's a minimal cost. Also, make sure you've set "module": "esnext" in compilerOptions in your tsconfig.json - setting "commonjs" will result in console error reg. shared module not being available.

EDIT: One more (cosmetic) issue is, that VSCode's built-in TS server doesn't seem to recognize the "include" part of tsconfig.json resulting in marking the dynamic import in parent app as error (Cannot find module 'app2/App' or its corresponding type declarations) but the build works fine. If anyone knows a fix for this, I'd be ever so grateful.

malcolm-kee commented 3 years ago

Sharing a recent setup in my current company that is NOT monorepo (can't share code as it's not open-sourced, but happy to clarify!).

The key components are:

  1. npm org (to publish the exposes type, not the code)
  2. (Optional) A custom pipeline (like CRA) that abstract the webpack config so I can enforce similar conventions for all projects.

The convention that I enforce is that modules that are exposed must be in a src/exposes folder. Then, I programmatically set the exposes key of module federation plugin based on the file, e.g.

return new ModuleFederationPlugin({
  exposes: {
    './exposes/file1': './src/exposes/file1'
  }
})

I have a custom tsconfig.json file that only includes src/exposes folder. When it compiles, it will generate another folder like

type
/exposes
  /file1.d.ts
  /file2.d.ts
/components
  /...

Writing some custom code, I generate a package.json file in that folder with name @npmorg/<appName>, which can be published.

In the host application, when it import the remotes, always use the convention @npmorg/<appName>/exposes/<module> and also install @npmorg/<appName> package (which only consists of type definition).

Everytime the remote app change its interface in exposes file, it just need to regenerate the type definition folder and publish it and the host just need to update that package to get the latest type definition.

ruanyl commented 3 years ago

Sharing my setup:

For the host which exposes modules

  1. I created a dts-loader which will emit and collect the .d.ts file. And also it generates the entry .d.ts file based on the exposes.
  2. Then I create a tarball(.tgz file) for the emitted types & entries. And I deployed this tarball with the application's statics, for example, to http://localhost:9000/app-dts.tgz

For the host which requires remotes

  1. I created a webpack plugin WebpackRemoteTypesPlugin which will download and unpack the tarball from remote automatically when running webpack.

More details can be found here: https://github.com/ruanyl/dts-loader Comments & feedbacks are welcome :)

adrianbw commented 3 years ago

For those of you creating .d.ts files, do you also have enums? I tried that approach earlier this year and found that my runtimes would throw up because enums need to be compiled to (extraordinarily weird) JS. Maybe I was doing something wrong?

arshita04 commented 3 years ago

I have a file in my remote which is exporting enums and interfaces and i need to consume this in my host application. My build keeps failing as either it fails with the error: / not declared and when i declare it, it fails with the error: not being used and at the place where i am using it it throws the error: cannot use namespace.

alexis-regnaud commented 3 years ago

Sharing my setup:

For the host which exposes modules

  1. I created a dts-loader which will emit and collect the .d.ts file. And also it generates the entry .d.ts file based on the exposes.
  2. Then I create a tarball(.tgz file) for the emitted types & entries. And I deployed this tarball with the application's statics, for example, to http://localhost:9000/app-dts.tgz

For the host which requires remotes

  1. I created a webpack plugin WebpackRemoteTypesPlugin which will download and unpack the tarball from remote automatically when running webpack.

More details can be found here: https://github.com/ruanyl/dts-loader Comments & feedbacks are welcome :)

Does this solution work well with Polyrepo as well?

OriAmir commented 3 years ago

More then 1 year from the day that issue was open , do we have some good solution how we can resolve that issue and make typescript work well with the plugin ? Tnx

@ScriptedAlchemy

ScriptedAlchemy commented 3 years ago

I'm not a TS user so I'm no help here.

dgpt commented 3 years ago

Sharing my setup:

For the host which exposes modules

  1. I created a dts-loader which will emit and collect the .d.ts file. And also it generates the entry .d.ts file based on the exposes.
  2. Then I create a tarball(.tgz file) for the emitted types & entries. And I deployed this tarball with the application's statics, for example, to http://localhost:9000/app-dts.tgz

For the host which requires remotes

  1. I created a webpack plugin WebpackRemoteTypesPlugin which will download and unpack the tarball from remote automatically when running webpack.

More details can be found here: https://github.com/ruanyl/dts-loader Comments & feedbacks are welcome :)

I was able to get this working in a polyrepo. One thing that tripped me up pretty hard was your exposes must explicitly reference the files you're exposing.

  exposes: {
    './theme': './src/themes/index.ts',
    './helpers': './src/helpers/index.ts',
    './constants': './src/constants/index.ts',
  }

This will not work:

  exposes: {
    './theme': './src/themes,
    './helpers': './src/helpers',
    './constants': './src/constants,
  }

Opened up this issue so hopefully it can be improved: https://github.com/ruanyl/dts-loader/issues/7

Also, I was able to automate the archiving by using tar-webpack-plugin, so my types are generated and synced whenever I npm start - pretty neat!