Using raw esm is fine for most use cases, especially content-based sites like blogs or marketing sites, and it's even better for sites that ship minimal JS.
This does leave performance completely in the hands of users and their source directory layouts though. ie: if you want to ship a single file for a page-wrapper, you can write it as a single file (or bundle it yourself before feeding it to toast). Both of these have some drawbacks.
One example of drawbacks for single-file is my own page-wrapper, which is approx 1700 lines at the moment. This works just fine (aside from my own mistakes unrelated to toast architecture, such as including amplify.js on every page, large inline svgs, tailwind increasing number of lines, etc), but it's also easy to treat this as a typical project that people might want to split up.
The dep graph should include js, css, images, etc as first-class dependencies
Compilation will have happened before serving. JS-whatever to browser-runnable ESM.
Tying it up with your serving infra gives you the opportunity to automatically do all the preload prefetch header whatever tricks you’ve been told are great, it without needing to craft that shit by hand.
Current State
We already have implicit dependency graph data through salsa, so we should start by making that more explicit and tracking dependencies by returning more interesting datatypes.
We could use daggy, if we're confident that dags are enough, to maintain an explicit dependency graph.
Open questions:
there any platform support for packing ESM into a single file, but treating it as individual modules?
in rust-for-js I mention a project that seems to be trying to achieve this kind of packing:
The new part of the approach is to inline multiple raw ES modules (including the uncompiled import/export statements) into a bundle file, and have the browser be able to to load each as an individual module.
Blobs. With import maps, it’s easy. But without import maps it requires rewriting import specifiers at runtime, which is what I’m doing. Implementation approach does not require in-browser parsing
Joel Denning
can the dependency graph itself be fit into salsa?
That is, when in development/watch mode, can we accurately update the explicit dependency graph?
Where are how is this applied?
Sunil mentions feature flags. feature flags could be used at prebuild time with some extra effort, or at "serverless render" time. Flags seem like a second step on top of esm packing, so leaving this open for now.
Even for static sites, we could ship an edge function that does the recombination. Netlify has support for this already, as does cloudflare, etc. Notably we may have to do import-rewriting on-the-fly due to lack of import map support in browsers today.
An edge function could also inject preload/etc headers if we're doing this, addressing #56
CSS
We don't have postcss support yet, but #18 is tracking that. In the meantime people are using react-helmet to include css files in head, typically in page-wrapper. Ideally we would move to a more statically analyzable method so as to determine css dependencies. There has also been some dissatisfaction with react-helmet itself, so revisiting that API itself is possible if it supports our goals here.
Using raw esm is fine for most use cases, especially content-based sites like blogs or marketing sites, and it's even better for sites that ship minimal JS.
This does leave performance completely in the hands of users and their source directory layouts though. ie: if you want to ship a single file for a page-wrapper, you can write it as a single file (or bundle it yourself before feeding it to toast). Both of these have some drawbacks.
One example of drawbacks for single-file is my own page-wrapper, which is approx 1700 lines at the moment. This works just fine (aside from my own mistakes unrelated to toast architecture, such as including amplify.js on every page, large inline svgs, tailwind increasing number of lines, etc), but it's also easy to treat this as a typical project that people might want to split up.
Sunil includes some interesting points, especially if we pursue any serverless applications.
summary:
Current State
We already have implicit dependency graph data through salsa, so we should start by making that more explicit and tracking dependencies by returning more interesting datatypes.
We could use daggy, if we're confident that dags are enough, to maintain an explicit dependency graph.
Open questions:
there any platform support for packing ESM into a single file, but treating it as individual modules?
can the dependency graph itself be fit into salsa?
That is, when in development/watch mode, can we accurately update the explicit dependency graph?
Where are how is this applied?
Sunil mentions feature flags. feature flags could be used at prebuild time with some extra effort, or at "serverless render" time. Flags seem like a second step on top of esm packing, so leaving this open for now.
Even for static sites, we could ship an edge function that does the recombination. Netlify has support for this already, as does cloudflare, etc. Notably we may have to do import-rewriting on-the-fly due to lack of import map support in browsers today.
An edge function could also inject preload/etc headers if we're doing this, addressing #56
CSS
We don't have postcss support yet, but #18 is tracking that. In the meantime people are using react-helmet to include css files in head, typically in page-wrapper. Ideally we would move to a more statically analyzable method so as to determine css dependencies. There has also been some dissatisfaction with react-helmet itself, so revisiting that API itself is possible if it supports our goals here.