Open dgp1130 opened 11 months ago
One other slightly crazy idea I had was whether or not route rendering could be parallelized and cached at the Bazel layer.
It's relatively easy to parallelize async render calls of routes. I say "relatively", it's actually a more complicated than you might think with generators. But ultimately if you have one prerender_pages
target which renders all the routes you're still gonna wait a long time for apps with a large number of routes.
Instead, I was imagining "sharding" the prerender_pages
target, just like test sharding. Where prerender_pages
could generate multiple prerender_resources
actions, each one would invoke the same binary with some arg which tells it to invoke the render
function of only a subset of the routes. Then all the outputs can be merged together in a web_resources
target. This could be done just by dividing the number of routes by the number of shards, or could be done by route paths such as:
prerender_pages(
name = "site",
# ...
route_shards = [
[ "/blog/**" ], # Shard 1 renders only the blog posts.
[ "/help/**" ], # Shard 2 renders only the help pages.
[ "/**" ], # Shard 3 renders everything else.
],
)
This would probably need to be in a new router-aware rule like prerender_routes
instead of prerender_pages
. While this parallelizes at the process level, it doesn't help much with caching because each shard needs to run the same binary. So if you edit the rendering of a help page, you still need to render all the blog posts. I suspect this could be improved if we moved forward with file-based routing.
If we instead define the API as:
prerender_routes(
name = "site",
# ...
routes = [
"//path/to/site/www:home",
"//path/to/site/www:about",
"//path/to/site/www/blog",
"//path/to/site/www/help",
],
)
Then if each routes
target exposed some standard convention, such as a *.route.ts
file with a default export of a router config:
export default {
label: 'Home',
render: () => { /* ... */ },
};
Then we could actually make each sharded action only depend on the files needed to render! The shard which renders the home page doesn't need to have a dependency on the :about
target at all and can be cached.
One challenge with this is the requirement:
If a page attempts to render a navigation UI which lists all the pages, then you need to know all page URLs up front before you can render a single page.
This does mean that the home page still needs to know that about exists, even if it isn't rendered. Either we need to design the *.route.ts
convention to not depend on the rendered component or we'd need to statically analyze the route to extract this information without building and running it. Static analysis might also be tricky given that we won't be able to transpile the route to JavaScript without building all its dependencies. We'd need to either:
I think it's a cool idea at least and one worth exploring after initial routing support.
So I was originally expecting that
@rules_prerender
wouldn't really need a routing solution because it would be generic enough to support any kind of site routing. Theexport default
entry broad is powerful enough to mostly do whatever users could want to do. However, while working on the docs site, I've changed my mind for a couple reasons.I'm working on a proof-of-concept directly in the docs site, but I think upstreaming a more comprehensive implementation would be very useful. Currently this looks something like:
The exact shape of this is TBD, but some requirements to consider based on this use case:
/products/:id
.