Static rendering all the routes, even the parameterized ones

[johnnysprinkles]

This is my exploration into the idea of static rendering all the routes such that they can be served in a totally language agnostic way. This is of course easy to do already for routes with no route params, but not so simple for the parameterized routes.

Why would I want to pre-render a parameterized route, where all the data comes from the clientside XHR call? Aren't I just prerendering a blank page? Not at all my friend, you prerender as much as you can which would typically include at least the global nav bar. Having that appear instantly instead of merely quickly upon hydration makes all the difference in giving your site an app-like feel.

And in my case it's especially important since I'm not using client-side routing, so each page load needs to be as fast-feeling as possible. Why no client-side routing? Imagine a large site with 1000 pages, an in-house project spanning many years and having many separate teams. Building this as a single app is bound to result in slow build times, so you can split it up into 10 Svelte projects where each has 100 pages (all behind a reverse proxy). But as long as you're splitting it at all it makes sense to just split it completely with full page reloads on every transition. Now you can reproportion ownership fluidly as you see fit, and most importantly, you can iteratively migrate page by page to some new framework. Perhaps whatever the hot framework of 2025 is. This a whole separate topic I could say a lot about but let's leave it at that.

So anyway, the '*' you pass to kit.prerender.pages only does the parameterless routes. A first attempt at this might be to add another route in there that has garbage for the param since it won't even be looked at from the serverside, and updating your code that runs onMount to only look at window.location instead of the page store.

I put together a repo to try out some ideas, at https://github.com/johnnysprinkles/static-all-the-things

Setup

The first commit is just setting things up with a small constellation of services to make a more realistic prod-like setup. There's SvelteKit, there's a simple API server, and there's a web server that serves the adapter-static compiled assets off of disk. Just to ensure we have a complete break between build time and runtime the web server is written in Python. Could be anything (for me it'll probably be Java or Kotlin), but all the matters is it's not Node.

On the SvelteKit side I added two routes related to airplanes, one is a parameterless List page, and one is a detail page that has an [id] in its route.

https://github.com/johnnysprinkles/static-all-the-things/commit/b06f612a31b4009e5a44a4b05b97f7c7cffd37f1

This example just gives you a 404 for the parameterized routes.

First pass

https://github.com/johnnysprinkles/static-all-the-things/commit/3be17a2a84147cdc516f83c756fd72ebe021ce65

  pages: [
    '*',
    '/airplane/XXX',
  ],

This manually adds preprender paths with garbage for the parameters, and it does work. I can run it and my dynamic pages function. It's all hacked up and manual though, we can do better.

First attempt at interpolating

We'd really like to keep using the store, which should be a source of truth for route params and other request-specific data, instead of going around it to inspect the window.location. So instead of passing garbage for the params when server-side rendering, what about if we pass something meaningful? Such as just the param name in square brackets? (I know, this is kind of overloading the meaning of square brackets, but it's just a start).

If we do that, we can just have the web server replace e.g. [id] with the actual ID. A hacked up and manual version of this is in:

https://github.com/johnnysprinkles/static-all-the-things/commit/5ed2e708ec1a355788b8bfb9deb24ff70d51b8e2

  pages: [
    '*',
    '/airplane/[id]',
  ],

More automated way of interpolating

We can tidy things up a bit by taking advantage of the manifest.js file, which knows both the names of all the route params (in JS comments) and the regexes that match, e.g.

export const routes = [
    // src/routes/index.svelte
    [/^\/$/, [c[0], c[2]], [c[1]]],

    // src/routes/networking/addresses/list.svelte
    [/^\/networking\/addresses\/list\/?$/, [c[0], c[3]], [c[1]]],
    ...

So the next version of this reads and parses that:

https://github.com/johnnysprinkles/static-all-the-things/commit/22e3e98149876364d8a4317e9557f727e2287021

It would be nice if we had routes presented in a language agnostic way, such as a routes.json file that lives next to the manifest. A possible feature request.

Once we have that we can fill in the page store via regex slice and dice. This would be a little easier if the page store data was pulled out of that structure (and also perhaps surrounded by special comment tokens). So for example we currently get this:

<script type="module">
  import { start } from "/./_app/start-aec04e6a.js";
  start({
    target: document.querySelector("#svelte"),
    paths: {"base":"","assets":"/."},
    session: {},
    host: location.host,
    route: false,
    spa: false,
    trailing_slash: "never",
    hydrate: {
      status: 200,
      error: null,
      nodes: [
        import("/./_app/pages/__layout.svelte-1240a5ff.js"),
        import("/./_app/pages/airplane/[id].svelte-a58c7efe.js")
      ],
      page: {
        host: location.host, // TODO this is redundant
        path: "/airplane/[id]",
        query: new URLSearchParams(""),
        params: {"id":"[id]"}
      }
    }
  });
</script>

But if it were more like this it would be easier for any language that can emit JSON to fill it in:

<script type="module">
  let pageStoreData = /** PAGE_STORE_START **/ {
    host: location.host,
    path: "/airplane/[id]",
    queryString: "",
    params: {"id":"[id]"}
  }/** PAGE_STORE_END **/;

  import { start } from "/./_app/start-aec04e6a.js";
  start({
    target: document.querySelector("#svelte"),
    paths: {"base":"","assets":"/."},
    session: {},
    host: location.host,
    route: false,
    spa: false,
    trailing_slash: "never",
    hydrate: {
      status: 200,
      error: null,
      nodes: [
        import("/./_app/pages/__layout.svelte-1240a5ff.js"),
        import("/./_app/pages/airplane/[id].svelte-a58c7efe.js")
      ],
      page: {
        host: pageStoreData.host,
        path: pageStoreData.path,
        query: new URLSearchParams(pageStoreData.queryString),
        params: pageStoreData.params
      }
    }
  });
</script>

So that's where things are at now, this will be an ongoing exploration.

[benmccann]

It's not clear to me what you're requesting. Do you have a concise way of stating it? Parameterized routes are currently prerendered, but must be either linked to or specified as an option

[johnnysprinkles]

Yes, parameterized routes are pre-rendered but that works best for a set of known values. Pre-rendering routes that have wildcard values where the values aren't known until request time isn't that well supported at the moment. It can be done (that's what my linked demo shows) but requires some fairly ugly parsing of the HTML artifacts. I think there might ultimately be 3 feature requests to make it smoother:

1. Exposing the routes from manifest.js as a JSON file
2. Factoring the page store data from the start() call out to make it easier to replace at runtime, and/or inherit from a global variable
3. Some new special syntax for the kit.prerender.pages array, something similar to the '*' but to specify to prerender all parameterized routes with placeholders.

It may be the case that what I'm trying to do is so uncommon that it doesn't belong in SvelteKit but in some other framework built on Svelte. Wanted to see if there's any interest.

[johnnysprinkles]

Also this runtime server-side interpolation into the HTML file, that is only one way to approach it. It could just include a copy of the route parsing logic in there. Then you could still serve your HTML files off a CDN and enjoy routes with placeholders. So in your start() call instead of:

  page: {
    host: location.host, // TODO this is redundant
    path: "/airplane/[id]",
    query: new URLSearchParams(""),
    params: {"id":"[id]"}
  }

You might have essentially a slice of the manifest that relates to that particular route, i.e. the regex with capture groups, and list of positional route param names. So like:

  function parseFrom(path) {
    // Parses using regex for this route and route param names
  }

  page: {
    host: location.host, // TODO this is redundant
    path: location.pathname,
    query: new URLSearchParams(location.search),
    params: parseFrom(location.pathname),
  }

I need to make a branch on my demo repository and explore this. The advantage of this is it could be done as a post build step, or even just part of the regular build if it was integrated into SvelteKit, instead of needing per-request interpolation.

I don't want to add to your open bug count, you can resolve this if you want, people can still find it in the history.

[benmccann]

This seems over-complicated. You just need to create a directory page which generates a link for each parameter value

[johnnysprinkles]

The values are infinite though.

[johnnysprinkles]

Or I should say arbitrarily large, and constantly changing.

[johnnysprinkles]

Basically I've got this pretty sweet prototype I'm building, it's way way faster than our extra-large Angular app even though it's doing full page reloads on every transition. I'm currently just serving it with npm run preview, but thinking about how I'm going to actually productionize it.

[johnnysprinkles]

I'm on discord as "jpsimons" just FYI.

[benmccann]

You can't prerender an infinite number of pages because the pretenders process would never end. You can prerender arbitrary / changing pages though by generating the links in the directory page accordingly

[johnnysprinkles]

I'm trying to pre-render one page, but one page that's parameterized, where the parameter is filled in at runtime from the path.

[benmccann]

If you can provide a list of all possible values of the parameter from a database or some other source you can prerender that page. If there are an infinite / arbitrarily large number of possible values and you cannot provide a list then you can't prerender now and never will be able to with any software because that's impossible.

[johnnysprinkles]

Sorry I think I'm explaining more but not explaining better... these are pages that have no load() function, all data is fetched from the client-side. But all the surrounding bits of the page that aren't data fed can be pre-rendered so they pop in immediately, as fast as the browser can deserialize HTML and turn it into DOM, which is effectively instantly. I might have a global nav, some kind of subnav, a left nav bar, maybe a footer, and definitely the spinner. The spinner will reset itself upon hydration, but hopefully that will be resolved after a roll forward discussed in https://github.com/sveltejs/svelte/issues/4308

[benmccann]

Ok. So just prerender that page then. I don't understand what the problem is. This whole thread is indecipherable to me. You might try asking for help on Discord instead

[benmccann]

Ohhh, I think I finally figured out what you're trying to say. You want to prerender a page where the route parameter is not used in load but only client-side? And prerendering is skipping that page because it doesn't have a notion that a parameter can be blank during prerendering because it's unused on the server side

[johnnysprinkles]

Exactly, my write-up was too wordy! Pre-rendering with [TODO] deferred values, deferred until runtime.

[johnnysprinkles]

So I made up a scheme for it, and wanted to share, but it could be more streamlined and more integrated into SvelteKit if we wanted. The one thing that makes me nervous is the svelte file path in the manifest.js is just in comments, I need those and hope they don't go away.

[benmccann]

This thread is really long and confusing. I'm going to close it and file a new issue referencing this one with a clearer description

[tw1t611]

Hey guys, I came across a similar problem. For me routing on production does not even work, if I refresh the page. However, on dev it works.

Here is the setup (sliced)

filestructure

.
├── __error.svelte
├── index.svelte
├── __layout.svelte
├── shop
│   ├── [accessory].svelte
│   └── index.svelte
├── [...slug].svelte

As you can see, I am using a wildcard route [...slug] on the first directory level. It might conflict with shop sometimes, but thats just a side note for now.

[...slug] takes the "path" of a page and queries its content from a headless cms. When building with adapter-static build/_app/pages/[...slug].svelte-*.js was generated. This way the routing does not work.

When creating the routes manually (slug1, slug2...), they get properly generated in build folder and routing works on production

Here you can check the behavior

1. Goto https://ari-motors-v2.vercel.app/
2. Click on "ARI Personenfahrzeuge" -> url changes to https://ari-motors-v2.vercel.app/ari-personenfahrzeuge and content loads
3. Refresh the browser -> url stays the same, but content of / was loaded

Is there a way to work around this, somehow specifying the routes somewhere? I really can't add all the routes as files manually, which the content creators create and they a changing constantly. Even if I did, one change in these files would cause me to do the work on all page files.

My two cents on the topic

Did you think about crawling the site for links you could possibly prerender? Like

1. Fetch and prerender /
2. Which links can I find here? -> Found /en
3. Let's try to prerender /en -> 200 ok, nice
4. Which links are on /en? Man, that will take some time, luckily I am a computer.

To be fair with you, build time will drastically increase. However that's not a big deal for me personally.

[johnnysprinkles]

What does your kit.prerender.pages look like? If your slugs are constantly changing, keep mind that you can still pre-render of course but any parts of the page that are data-fed by request specific information will have to be client-rendered. If the page in its entirety is data-fed by request specific params, probably no point in static pre-rendering at all.

I'm a little unclear on your ultimate hosting solution. If it's Vercel, why not use adapter-vercel instead of adapter-static? adapter-static would be for a simple static file server, or as I'm pushing forward, a completely custom server runtime (the latter of which could allow query strings, path params, rest params, etc).

[tw1t611]

Hi, thanks for your reply. As you already hinted, adding the routes to kit.prerender.pages solved the prerender/routing problem.

svelte.config.js

    prerender: {
      crawl: true,
      enabled: true,
      force: false,
      pages: [
        "*",
        "/ari-personenfahrzeuge",
        "/elektrotransporter-ari-901/kofferaufbau",
        // other routes generated in [...slug]
      ],
    }

To answer you questions: I am using a headless cms, where some content creators create pages/posts. These are the routes that constantly change. As there is no data that changes at runtime adapter-static should get me the best results. Furthermore, I only want to publish new posts, when the site builds. adapter-vercel seems to SSR at runtime on vercel functions. Personally, I am a fan of using the things as they are meant to be, so I would prefer not to build a custom server runtime.

Just out of curiosity, the config states crawl: true, so why wasn't /ari-personenfahrzeuge prerendered? The link is in the menu load()ed at build in __layout.svelte.

Nonetheless the config works absolutely fine. I'd love a function in future, which crawls links from pages and prerenders them. However, I don't want to complain for svelte kit already is the best SSG and svelte is the fe framework so far. :)

[johnnysprinkles]

I think I understand... new pages come into existence constantly, but you only them to be published on the next build/deploy? And you don't want to have to list them all out in the pages array, you want them to be found by crawling? I suspect you found a bug, and that the use of rest params in the path is confusing the system?

[johnnysprinkles]

I'd try this just to test: Add a parameterized route without rest params, such as /elektrotransporter-ari-901/[name].svelte and update your code to look at 'elektrotransporter-ari-901/' + page.params.name instead of just page.params.slug, and see if that is crawlable, just to compare with the rest params case.

[johnnysprinkles]

Maybe crawling stops at routes with params in them, by design, to avoid pre-rendering your entire database. Maybe you also aready mentioned such in your description, I should read more carefully. Sounds like we might want a new option for crawlAllowDynamicRoutes or something.

[tw1t611]

I think I understand...

Yes, that's right. The bug I found was in regards to routing, when loading the page first. I assume you differentiate routing on first page load and when clicking a relative link on the site. So refreshing the page in the browser did load the wrong page content, even when the url stayed correct.

I'd try this just to test:

Edit: I did. Sadly the problem stays the same. So it seems like elektrotransporter-ari-901/[name].svelte does the same as [...slug].svelte did.

Maybe crawling stops at routes with params

In my case, I really want to prerender to entire database :D, or lets say all the pages/posts. I'd like crawlAllowDynamicRoutes.

[lsabi]

To be honest, I expected crawl: true in the config to behave, as crawlAllowDynamicRoutes. Since it crawls in order to build the static site, it is to be expected that all links that are encountered are rendered.

If someone doesn't want to pre-render the entire database, one should set the pages to be rendered in the config.prerender.entries. It's simpler to config a couple of pages that need to be rebuilt (or need to be built for the first time), than to list the whole content of a database (which may be thousands of pages).

These are my two cents from reading the documentation, as I can't find this problem mentioned anywhere in the docs.

[Frioo]

To be honest, I expected crawl: true in the config to behave, as crawlAllowDynamicRoutes. Since it crawls in order to build the static site, it is to be expected that all links that are encountered are rendered.

This! In my case I'm trying to set up a simple blog that renders pages based on markdown files. I really expected crawl to pre-render all linked pages, which it doesn't. I agree sometimes it would not be a good idea to render a whole database of entities, but there are cases where this would be greatly beneficial (I could have a completely static blog with a fantastic build process!).

[johnnysprinkles]

So I was actually thinking more about rendering a generic version of a dynamic route, and actually this literally just came up again at my current employer... I want to use adapter static to spit out some html files and an _app folder that I can drop in an existing web server setup (in my case it's Java/Dropwizard/Maven). You can actually pull this off just fine by adding a prototypical example for each parameterized route in pages, i.e.

pages: [
  '*',
  '/[id]'
]

Then do a little URL rewriting in your web server, something like:

<rewrite>
  <from>^/\d+$</from>
  <to>/[id]/index.html</to>
</rewrite>

The downside here is the realtime path params won't come through in the page store params, but you can simply inspect location.pathname yourself. However... if you navigate from say /123 to /456 it won't tear down and reconstruct your view, so really do need to do everything in the just recently added afterNavigate instead of onMount. I'd actually like to write a guide about all this stuff sometime.

[iham]

so this is still a thing to be done?

i am new to svelte/svelteKit, did a little research on other frameworks (gatsbyjs for example) and seek for the ability to create static pages from dynamic routes.

i too thought the crawl option would unfold all those dynamic routes [page].svelte to static html pages based on the result of an api result.

is there any solution to that? a different adapter? other settings/options on kit.prerender?

any help is appreciated

[benmccann]

You need to either link to each page somewhere within your app so that it will be crawled or you need to list each page in the prerender.entries option: https://kit.svelte.dev/docs#configuration-prerender

[iham]

@benmccann

i have an api route to fetch users (using faker.js - older version; not 6.6.6)

the result of that list is used by module load() in my index.svelte route and links each entry as a detail page using a dynamic route [user].svelte

the second api [user].js fakes additional data for that user. this data is again used by module load() in my [user].svelte route

npm run dev works perfectly fine, prefetching does its thing and data is shown on the list and user page. all fine.

but npm run build only renders my list page. including the correct links to the detail pages for each user -> so crawling should do its thing, but no detail page is statically created)

you mentioned to add prerender.entries which entries?

• my user list route? - that one is created
• my dynamic route for the detail page? - how?
• each user page (i don't know as this is dynamically built)?

maybe i have a missconception of what adapter static and prerender does... i wanted to test the capabilities of sveltekit to do SSG.

[benmccann]

You would need to file an issue and provide a code repository that reproduces the issue. I'm not sure why crawling wouldn't work for you

[iham]

Was my next thought, to provide some reproducible code

I will do that and link it here

Thank you

On 01.02.2022, at 14:19, Ben McCann @.***> wrote:

 You would need to file an issue and provide a code repository that reproduces the issue. I'm not sure why crawling wouldn't work for you

— Reply to this email directly, view it on GitHub, or unsubscribe. Triage notifications on the go with GitHub Mobile for iOS or Android. You are receiving this because you commented.

[xpy]

Why is this closed? Now it is impossible to create a static page with dynamic params. I think it is not uncommon that you don't know your data on build time.

[xpy]

If I get it correctly and the problem is the same as mine:

You have a route users/ that is supposed to be static, and a database of users that is dynamic and is fetched on the frontend, and you want to be able to able to navigate to uses like users?user_id=80085, but you dont want to prebuild the users endpoints. But the build fails because you cannot use the url.searchParams for prerendered routes.

After very much research, I came to the solution to catch the error and return empty userId instead:

  export async function load({ url }) {
    try {
      return { props: { userId : url.searchParams.get('user_id } };
    } catch (e) {
      return { props: { userId: '' } };
    }
  }

That way the path is properly build and can be visited properly in both user\ and user\42

What else I tried and failed or didn't work for me:

• use path params like user/[id].svelte and then query like user/313. Hydration works fine but you cannot go directly to user/313 since it doesn't exist
• use hash params. You have to do add an event listener for the onhashchange and handle the reactivity yourself, also the history was kind of broken, it would push two states instead of one upon navigation.

[johnnysprinkles]

I'm not quite sure I understand... users in your example is pre-rendered so it's an HTML file sitting in your /build folder? In that case you'd need to do any "infiinitely broad aka parameterized" loading client-side i.e. onMount. You could actually have a mix of data that's fetch at pre-render time, such data wouldn't change very often since it only updates when you redeploy static assets, and dynamic data fetch that's done client-side.

I think your first bullet point is on the right track... although in my case at my current employer, our build/deploy system chokes on square brackets, maybe having something to do with bash escaping, so I used underscores instead. As far as going directly to users/313 that's up to your web server to figure out, adapter-static lets you "bring your own server".

[xpy]

@johnnysprinkles If you have a completely static site and no power over the server then you would just get a 404 on users/313 because it won't exist in the file system, users?user_id=313 would work though because the users/index.html would be there. As for the users/ url, it is up to you how you handle the page without a user_id.

I created a PR that should work on your example: https://github.com/johnnysprinkles/static-all-the-things/pull/1 I didn't run it but I trust it should work.

But anyway, I think that using the searchParams shouldn't be forbidden, it should be handled as if the params are empty and give a warning to the developer, or at least the option to give a default value.

[johnnysprinkles]

Oh I see, your web server is just a simple CDN so yeah, path params aren't an option. But your CDN will probably wildcard route all query-string suffixed URLs to the same filename. In that case, why use load() at all? Why not simply examine window.location.search from onMount?

[johnnysprinkles]

Oh maybe I see what you're saying, url passed to load() is clientside runtime dynamic unlike the page store? Let me investigate.

[xpy]

You cannot access searchParams onMount, it throws an error on prerender. You can though ( I found out later ) condition if the code is running on the browser or on prerender: https://kit.svelte.dev/docs/modules#$app-env and do the same thing I did with the try/catch but with an if(prerendering)

import {  prerendering } from '$app/env';
 export async function load({ url }) {
    if(!prerendering) {
      return { props: { itemId : url.searchParams.get('id') } };
    } else {
      return { props: { itemId: '' } };
    }
  }

And then render a placeholder page if there are no params.

And as you already said, the pages are indeed in a CDN so it should be just dump static html pages.

[ayazemregm]

I have read through all of the answers. Is there any update on this one? I have a folder structure like this:

─ src ── blog ─── +page.svelte ─── [blog_post] ──── +page.svelte

The thing is I have 0 references to blog_post page in blog page because in the blog page I get active blog posts from an API. I create links through slugs of the blogposts. Like "example.com/blog/blog-about-something". The problem I am having is when I run static generation it has 0 references to blog posts page because blogs page is generated dynamically on client side as well.

I want to generate my navbar, footer and other elements statically and use my custom js fetch function to fill the blog post data. When I run the build it says crawling was unable to find references to blog_post page. If I add a reference like <a href="/blog/test-post"> it finds the page but when I change the test post slug I get 404, which is expected. What I need is something like "blog/*" so I can serve same page all the time for anything that comes after "blog/". I can use "window.location" to complete my fetch function and if it is not found I can show my own 404. Only thing I want is to get my navbar, footer and other elements to be prerendered.

Sorry if this diverges from the main post of this thread but I think we have same problem.