embedding Flow components into TypeScript views does not work

[vlukashov]

If I follow the Creating an Embedded Vaadin Application Tutorial in order to embed a server-side Vaadin component into a TypeScript Vaadin view, my application does not work as expected:

• the tutorial tells me that I need to include the webcomponents-loader script and gives code snippets how to do that. These code snippets do not work. Moreover, the entire step of loading WebComponents polyfils is not needed because all browsers supported by Vaadin 15+ do support Web Components natively.
• If I skip the WC polyfils (or if I manage to find my way to load them), embedding still fails with a Uncaught TypeError: window.Vaadin.Flow.initApplication is not a function error:

ServerCounter.java:

public class ServerCounter extends Div {

    public ServerCounter() {
        add(new Label("Server-side counter"));
    }

    public static class Exporter extends WebComponentExporter<ServerCounter> {
        public Exporter() {
            super("server-counter");
        }

        @Override
        protected void configureInstance(
                WebComponent<ServerCounter> webComponent,
                ServerCounter counter) {
        }
    }
}

index.html:

<script type="module" src="web-component/server-counter.js"></script>

main-layout.ts:

render() {
  return html`
      <server-counter></server-counter>
      <slot></slot>
  `;
}

[vlukashov]

Workaround (by @⁠tomivirkki):

We had the same issue = Polymer (and the Vaadin components) got imported from two separate bundles: the one from docs-app and the one from Vaadin (Embedded views)

This is resolved by excluding all Web Components / Polymer related imports from the Vaadin bundle: https://github.com/vaadin/docs/blob/master/webpack.config.js#L35-L36

const fileNameOfTheFlowGeneratedMainEntryPoint = require('path').resolve(
  __dirname,
  'target/frontend/generated-flow-imports.js'
);
const filteredFileNameOfTheFlowGeneratedMainEntryPoint =
  fileNameOfTheFlowGeneratedMainEntryPoint + '-filtered.js';

// @ts-ignore
module.exports = merge(flowDefaults, {
  entry: {
    bundle: filteredFileNameOfTheFlowGeneratedMainEntryPoint
  },
  plugins: [
    function(compiler) {
      compiler.hooks.afterPlugins.tap(
        'Filter out external deps',
        compilation => {
          const original = fs.readFileSync(
            fileNameOfTheFlowGeneratedMainEntryPoint,
            'utf8'
          );

          // Exclude component imports which are included in the "bundle" module
          const filtered = original
            .split('\n')
            .filter(row => {
              if (row.startsWith("import '@vaadin")) return false;
              if (row.startsWith("import '@polymer")) return false;
              if (!row.startsWith('import')) return false;
              return true;
            })
            .join('\n');

          fs.writeFileSync(
            filteredFileNameOfTheFlowGeneratedMainEntryPoint,
            filtered
          );
        }
      );
    }
  ]
});

[vlukashov]

More context in a slack discussion

Flow embedding currently works in a way that it creates its own vaadin-export bundle out of the generated-flow-imports file. In webpack terms that's an entry, i.e. a root of a separate dependency graph.

By design, webpack entries are not expected to be loaded several into the same page - there is no dependency dedup between them. This leads to problems when embedding Vaadin into any app that also uses Polymer, including V15 TS apps.

If I create a V15 TS app that does not use Polymer (i.e. does not use Vaadin components), embedding server-side Vaadin components into it works just fine as described in the embedding docs. However, as soon as I add a <vaadin-button> into the client-side bundle, embedding breaks because now Polymer dependencies end up both in vaadin-bundle and vaadin-export bundles.

But that's not all. Apparently, in addition to the npm dependencies conflict that comes with embedding there is also a conflict between the server-side WebComponents bootstrapping code and the AppShell bootstrapping code. Or that's what I would guess from the Flow.initApplication is not a function error in the browser logs.

[haijian-vaadin]

Note, the steps to import a server-side web component do not need to be the same as embedding into a general application (e.g. JSF). To embed a server-side component, a user could import the web component by import 'xxx/server-counter'; or sth similar.

[platosha]

I have reproduced the issues above and several others. Let me first try to list and categorize them.

1. Remote import issues

Flow’s web component export scripts are served using a request handler. Importing that as a remote script (e. g., from a URL) is hard for Fusion TypeScript views:

• import '/web-component/server-counter.js'; fails resolution at compile time in TypeScript and webpack, as they both attempt to resolve and load the script
• with //@ts-ignore and /* webpackIgnore: true */ workarounds for the above applied, importing fails at runtime when the loaded bundle attempts to discover document.currentScript, whereas with webpack, the consumer view script’s execution context does not provide it
• adding <script type="module" src="/web-component/server-counter.js"></script> to the view template in the render() does not work because of the undocumented security feature in lit-html, which removes script tags, see: https://github.com/Polymer/lit-html/issues/759

As of now, the only way to load the remote web component export bundle in Fusion is to add a <script> tag to the document, either in the index.html template, or dynamically using DOM APIs.

2. Duplicate web component dependencies

As already mentioned above, any duplicate web component dependencies, such as using <vaadin-button> or Polymer in both embedded and consumer applications, result in a conflict, because there is no deduplication between bundles, and because web components are registered in the global customElements registry.

Global registry for custom elements is a limitation in the web platform, which is not likely to be relaxed anytime soon. There is a proposal for Scoped Custom Element Registries, with no implementation or shim available yet.

This issue is less specific to Fusion, and likely manifests in Flow applications embedding other Flow applications too.

3. Conflict in global Vaadin namespace

In addition to errors from web component registration conflicts, there are also errors likely caused by conflicts of declaring and using global Vaadin namespace in both consumer and embedded application:

• Uncaught TypeError: window.Vaadin.Flow.initApplication is not a function
• Uncaught TypeError: $wnd.Vaadin.Flow.registerWidgetset is not a function

4. Documentation

Following the Creating an Embedded Vaadin Application Tutorial with Fusion application is hard. The tutorial is generic and does not tell about the above issues, and there is no separate Fusion specific article.

---

Now on to solution ideas. While issues (3) and (4) are more straightforward to address, we have to consider which way to go about (1) and (2). Here are some options:

Add generated sources for importing exported web components

Example import:

import 'Frontend/generated/web-component/server-counter';

This solves both (1) and (2). Here the import becomes local instead of remote, and web component dependencies are managed in a regular way by the consumer application.

Pros:

• Allows for type checking (if TypeScript definitions are generated also)
• Same usage as with regular web component dependencies in a Fusion view

Cons:

• One more frontend generation build task.
• Solves only local embedding, i. e., exported Flow components into Fusion views of the same application repository. Some additional effort (emitting and using Node compatible package metadata and index) is required to enable source import of exported web components form separate dependency applications.
• Does not help with remote embedding (e. g., web components exported from network resources, separately deployed)

Use Module Federation in webpack

Webpack has introduced the Module Federation concept, which is targeted at the same use case (Micro-Frontends). This could solve (1) and (2) with the following ideas:

• The export bundle exposes exported and dependency modules for the consumer
• The consumer application is configured to load specific modules at runtime from a remote bundle instead of bundling them at compile time
• At the same time, the consumer application bundle can override specific remote exported modules, if necessary

Pros:

• Allows for remote embedding and separate deployment

Cons:

• Depends on webpack 5 on both the exporting and the consumer side
• Additional webpack configuration is needed on both ends
• Cannot share type definitions for TypeScript in consumer application, for example, for user-configured properties in exported components

Developing own custom solution for (1) and (2)

We could also tackle (1) and (2) as separate issues, with a custom solution for each:

• Simplify loading remote bundles (1) with some import conventions, e. g., import 'Remote/web-component/server-counter';
• In addition, for (2), solve duplicate web component dependencies in some way:
  • For inspiration, @open-wc/scoped-elements makes automatic renaming of components under the scope. This is not directly applicable in our case though, because Vaadin components are self-registering.
  • Alter the bundling in some way to deduplicate dependencies. For example, use a global replace when bundling to filter-out dependencies from the export bundle (see the workaround above), or to replace registration of the web components in the export bundle with a hook that allows control from the consumer application.
  • Wait for a shim or implementation of scoped registries

As with any custom solutions, there are cons of more development and maintenance effort, as well as compatibility risks.

[vlukashov]

Thanks for a great summary, @platosha! 🥇

Embedding within one app

One use case for embedding server-side web components into TypeScript views is hybrid apps. The currently supported hybrid approach is to combine Flow (Java) and Fusion (TypeScript) UIs keeping them in separate routes. Sometimes a more granular combination may be handy: one may want to include an existing server-side Java component into a TypeScript view. This would be the case when an existing Flow-based app is updated to Vaadin 15+ and developers want to keep using existing (Java) components in TypeScript views as well.

In this case the option A (add generated sources for importing exported web components) looks like a good fit. The breakdown of work for the option A is as follows

• change the vaadin maven plugin so that for each exported server-side web component a frontend/generated/web-component/server-counter file is generated at the build time
• create a custom bootstrap handler / update the existing WebComponentBootstrapHandler to work with the new import semantics
• consider unification between importing a single web component and importing the Flow serverSideRoutes route. In both cases there is a web component with content and behavior fully controlled by server-side Java code
• update the "embedding" docs

Here is how DX would look like: ServerCounter.java:

public class ServerCounter extends Div {

    public ServerCounter() {
        add(new Label("Server-side counter"));
    }

    public static class Exporter extends WebComponentExporter<ServerCounter> {
        public Exporter() {
            super("server-counter");
        }

        @Override
        protected void configureInstance(
                WebComponent<ServerCounter> webComponent,
                ServerCounter counter) {
        }
    }
}

main-layout.ts:

import 'Frontend/generated/web-component/server-counter';

render() {
  return html`
      <server-counter></server-counter>
      <slot></slot>
  `;
}

It will have the limitation that the embedded server-side components cannot be in a different project - embedding works only within one Vaadin project.

Embedding across apps

Another use case for embedding server-side web components into TypeScript views is portal-like apps. The current support for portals includes portlet and OSGi support. This has a limitation that all portlets in a portal need to use the same version of Vaadin, and TypeScript views are not supported at all. No support for TypeScript views could be a road block for customers looking to adopt TypeScript views in portlet / OSGi environments.

In this case a variation of the option C (using an own in-house solution for splitting bundles) looks like a good fit. In fact, the portlet / OSGi support is made possible by a custom bundling logic where Polymer and other frontend dependencies are deployed as a single shared bundle and re-used by all portlets in a portal. When designing support for TypeScript views this existing implementation should be taken into account.

[vlukashov]

Another use case for cross-app embedding: gradual migration from an older Vaadin app to Fusion: want to embed parts of the old app inside the new app, without putting the old and the new app into one project.

[haijian-vaadin]

One extra task to consider is that currently the web component is generated into target/frontend folder, should we keep the file there? or change the logic to use the file in the generated/frontend folder.

The component would have some communication API to the consumer view, e.g. is it ready. since the web component is normally loaded asynchronously.

A first rough estimation of the effort is 2 persons 2-4 sprints.

[Artur-]

A working example can be found here https://artur.app.fi/embed-fusion-flow/. Code here https://github.com/Artur-/embed-fusion-flow

[Legioth]

Proper embedding support has been implemented and documented for Vaadin 24.4