Needing a way to name/identify additional entrypoints

chancancode commented 1 year ago

:warning: This issue evolved into something else, https://github.com/embroider-build/embroider/issues/1589#issuecomment-1765492306 is probably the best summary of the missing feature, but I'll keep the original issue description around. It is one of the approaches I tried initially to ultimately accomplish what is needed in that comment.

Not sure if this is supposed to be supported, but say you have:

// app/count.js

window.count ??= 0;
window.count++;
console.log(`count.js loaded ${window.count} times`);

// app/my-extra-entrypoint.js
import "./count";

...and in app.js you add:

import "./count";
// ...everything else

and in index.html you add:

    <!-- ...everything else -->
    <script type="module" src="my-extra-entrypoint.js"></script>
    <script type="module" src="count.js"></script>
  </body>
</html>

Then, you'll see three distinct copies of count.js included twice into different chunks, and window.count will be 3 when everything is loaded.

By the way, src="./my-extra-entrypoint.js" seems to cause a different kind of trouble where it gets incorrectly rewritten to look for .embroider/rewritten-app/.my-extra-entrypoint.js – note the extra . at the beginning of the filename.

chancancode commented 1 year ago

Here is the reproduction: https://github.com/chancancode/embroider-html-entrypoint-bug/commit/943ea8b9df43376af59b590b5bd4a02f422d7b56

As discussed from the spec meeting last week, I think we all expect this to work, but somehow it appears to be treated as a completely distinct module tree by Webpack.

When splitting up the bundles, it is normal for Webpack to sometimes duplicate the same module into multiple chunks (e.g. when a module is needed by two different async entrypoints but not needed in the main bundle), but the module would still get the same moduleId so that the Webpack runtime can ensure the module is only ever loaded once at runtime.

For some reason, this is not the case here.

ef4 commented 1 year ago

Here is a reproduction of the same behavior in vanilla webpack: https://github.com/ef4/webpack-bug-example/

I don't think webpack anticipates ever putting multiple entrypoints on one page. You can make dependencies between entrypoints (our default tests entrypoint includes the default app entrypoint) and this works the way one would expect.

What are you trying to do? If your extra module is actually part of the single webpack build, it can be imported (including conditionally imported) from existing JS instead of being a separate HTMl entrypoint. If you extra module is build or distributed separately, then it's not going ever share module ids with the app anyway, unless you can make webpack module federation work.

chancancode commented 1 year ago

Hm, I see. Based on threads like https://github.com/embroider-build/embroider/issues/1305 I was expecting this to work.

I was thinking what we are doing is more akin to having app/index.html as the sole entrypoint, and using something like html-loader to discover the dependencies, which would make multiple <script type="module"> in the HTML (I suppose calling them "entrypoints" is a bit of a misnomer) more akin to multiple await import("...");, which does work the way I expected (sharing the same runtime chunk, moduleIds, etc).

I was hoping to replace some of the extraPublicTrees, currently produced with Broccoli, without having to rearchitect/refactor a bunch of other things. Essentially we have existing additional bundles that we produce and conditionally include/omit them in the HTML with a bit of help from the Rails server in production. These are manual code-splitting I suppose, but it's for optional features (admin panel, first boot setup wizard, etc) that the server knows whether is needed based on server side states (whether the logged in user is admin, whether this is the first boot, etc).

I was hoping to replace the manual tree constructions with something like:

<html>
<body>
  <!-- everything else -->

  <!-- begin admin -->
  <script type="module" src="./admin.js"></script>
  <!-- end admin -->

  <!-- begin wizard -->
  <script type="module" src="./wizard"></script>
  <!-- end wizard -->
</body>
</html>

...and have the server serving the index.html conditionally omit things between the magic comments as needed.

But I could do something else with slightly more effort if this is not meant to work.

void-mAlex commented 1 year ago

would the server outputting a global variable into the html that is then checked in your app code to do a guarded dynamic import of those modules not solve your problem? then you'd get the chunk split from main bundle, webpack would see it as a single entry point; should tick all your current stated needs

chancancode commented 1 year ago

would the server outputting a global variable into the html that is then checked in your app code to do a guarded dynamic import of those modules not solve your problem?

Semantically this is fine, but in terms of performance, it is not quite the same thing, as you would have to parse and evaluate the app.js before the browser even realize it needs to go and fetch the other bundle, wait for it to load/parse/etc.

What I am trying to accomplish isn't really about module federation per-se, it's really more to do with server-injected prefetching hints.

I don't think this is really that uncommon/unique to my situation either – image you have fully route-split app where every single route is split into a lazy bundle and dynamically loaded. You get a very minimal initial bundle with just the Ember stuff and bare bone router, etc, which kicks off the routing code, transition into the route and load the corresponding route bundle.

Semantically, this would works just fine. However, this may very well have introduced an initial load time regression, because the bundle for the code that is actually needed for the initial render is loaded much later than before.

Ideally what you would do to recover that performance is to have the server inject bundles it knows are needed into the index.html dynamically based on what it knows (in this example – it knows based on the URL; in my other example, it knows the user's locale, admin role, etc, based on the cookie session). Of course this is not something that Ember can do for you out of the box, but I think it's reasonable that Ember has some stable API to facilitate these kind of coordinations.

What I eventually found to have work well for this use case, that relies on the least amount of private implementation details, is to explicitly name these prefetchable bundles in the entrypoint config:

// app.js

switch (window.currentSession.locale) {
  case "en-uk":
    await import("./locales/en-uk");
    break;

  /* ... */

  default:
    await import("./locales/en-us");
    break;
}

if (window.currentSession.isAdmin) {
  await import("./static/admin");
}

// ...

// ember-cli-build.js
return compatApp(app, Webpack, {
  staticAppPaths: ["locales", "static"],
  packagerOptions: {
    webpackConfig: {
      entry: {
        "assets/bundles/locales/en-uk.js": {
          import: "./locales/en-uk",
          dependOn: "assets/my-app.js",
          runtime: false,
        },
        "assets/bundles/locales/en-us.js": {
          import: "./locales/en-uk",
          dependOn: "assets/my-app.js",
          runtime: false,
        },
        "assets/bundles/admin.js": {
          import: "./static/admin",
          dependOn: "assets/my-app.js",
          runtime: false,
        },
      },
    },
  },
});

<!-- app/index.html -->
<body>
  <script src="{{rootURL}}assets/vendor.js"></script>
  <script src="{{rootURL}}assets/my-app.js"></script>

  <!-- https://github.com/embroider-build/embroider/pull/1605 can be used to identify the chunks -->
  <script src="{{rootURL}}assets/bundles/locales/en-uk.js" data-bundle="locales/en-uk"></script>
  <script src="{{rootURL}}assets/bundles/locales/en-us.js" data-bundle="locales/en-us"></script>
  <script src="{{rootURL}}assets/bundles/locales/admin.js" data-bundle="admin"></script>
</body>

Not an ideal solution in terms of the boilerplate required and having to keep things in-sync in a couple of places, but as far as I can tell, this works great and does what I needed.

Ideally:

There would be a well-supported, less verbose way to do this, such as what I originally tried at the top of the issue thread
It would work* across different bundlers
There would be a way to refer to the implicitly generated route bundle/route entrypoint module

chancancode commented 1 year ago

We touched on this in the meeting today. It seems like @ef4 agrees this is a legitimate need/use case.

For the time being the snippet I have above works and it's probably good for someone to find it.

We could explore using the <script src=... type="module"> as a way to declaratively mean this.

Additionally, we may have to expose a way to refer to the virtual module for the route entry points like #embroider/routes/<name> that can be used in either the entrypoint config or in the <script src>.

That works for for me, though ideally I wouldn't have to put it in the index.html just to strip that out, and could just get a .json back from the build, but it's also not a very big deal.

chancancode commented 1 year ago

An update for those finding/following this thread, I think Discourse has settled on putting the entry point in the config and reading it back out using the stats plugin:

https://github.com/chancancode/discourse/commit/7c51dd43cd86c6af2156d7124983452577ecd532

I do still think it is important that we have some way to do this in a packager agnostic way. In fact, Embroider internally has the some similar needs:

needing to identify and preload the lazy chunks in fastboot
needing to identify and register the route-specific lazy chunks in a global, so that @embroider/router can find them

embroider-build / embroider

Needing a way to name/identify additional entrypoints #1589