Building a static website / Prerendering

[jstet]

https://github.com/sveltejs/kit/issues/1533

[jstet]

@ShirkahnW

[KonradUdoHannes]

Do we really need static rendering, what are the pain points exactly? If it is load time and directus availability we might get very far by moving as much as possible to the svelte server side rendering plus caching configurations for the directus api requests. In combination the backend might be able to cache api requests to directus and use them for multiple clients requesting the website. Also short unavailibities of the directus API would be covered by the cache.

[jstet]

There are no pain points currently, but there might be in the future. If we build the website to be static, we dont need to worry about directus at all once its built. This includes resource shortcomings of the infrastructure directus is hosted on. Hosting a static website is cheaper. The speed of static websites cant be beaten. As written in #40, we could host the site dynamically as a preview to check if the content written in directus is acceptable and then build it statically as a stable checkpoint.

[friep]

could we still do "dynamic elements" such as a filterable/searchable table with a static setup? Thinking of the project database...

if that's the case, i vote for static - given the kobotoolbox disaster (still down...) of the last couple of days i don't want to put @jstet or myself in a position where we desperately have to fix a directus instance..

[KonradUdoHannes]

A few notes on the progress:

• creating a static site will require to prerender all pages.
• when doing a non-static deployment, by default it is decided automatically which pages are prerendered. This can be made explicit by setting export const prerender = 'auto' in the top level +layout.js or +layout.server.js.
• auto prerendering does not render any pages of our website at this point.
• auto prerendering crawls the page to find routes, starting from any paths with no path parameter by default (we don't have any of these)
• the svelte.config.js can instead specify which paths to prerender. This also causes the crawler to expand from these starting paths
• Crawling has issues finding some links. In particular NavBar itmes that first have to unfold are not found
• Links to dynamically generated content like blog posts are not found
• since svelte.config.js is a regular .js file, we could programatically provide all paths for all our routs in order to cover prerendering for all all path parameters
• The Mitgliedsantrag Stellen page is problematic wrt. prerendering because it uses secretes which would be embedded in the page files if prerendered.

[KonradUdoHannes]

@jstet I think the last point from the list above is something that we conceptually have to discuss. I think if we don't find a solution we cannot use a static site, because we cannot expose the secrets.

In terms of performance (not cost) it might already be beneficial if we prerender many of the pages (as compared to non currently)

[jstet]

could we still do "dynamic elements" such as a filterable/searchable table with a static setup? Thinking of the project database...

if that's the case, i vote for static - given the kobotoolbox disaster (still down...) of the last couple of days i don't want to put @jstet or myself in a position where we desperately have to fix a directus instance..

I think we could somehow preload the data during the static build and then filter the data client side. Do you agree @KonradUdoHannes

[jstet]

@jstet I think the last point from the list above is something that we conceptually have to discuss. I think if we don't find a solution we cannot use a static site, because we cannot expose the secrets.

In terms of performance (not cost) it might already be beneficial if we prerender many of the pages (as compared to non currently)

Maybe we can just include a form on the website that sends a request to an external API that we build separately. This API could then include all the necessary secrets. I think turnstile also works without a secret client side. I feel like we should not give up on the advantages of a static website just for the membership application.

[KonradUdoHannes]

I'm not an expert on svelte, but as far as I understand we, we still get a lot of benefits from the SSR even if not the entire site is static. Nonetheless I'm completely open to make the entire site static if we manage. I just want to openly address these issues to reduce the risk of accidentally including secrets in a static side.

Regarding one of the original motivations for a static site, which seems to have been directus availability. I just want to point out that the main solution here is caching the directus data. Prerendered pages, can be setup to do this but its actually a separate requirement because they can also be set up differently. This is definitely a nice way to get the caching, but even if we don't manage to prerender a certain page, caching can also be setup explicitly to help with directus downtimes. We can keep this in mind in case the need for more forms on the website arises in the future.

[jstet]

The current directus setup (https://github.com/CorrelAid/CorrelIaC/blob/main/ansible/templates/docker-compose.yml ) already includes a cache but i set the caching time to be very low because otherwise changes in the cms are going live annoyingly late

[KonradUdoHannes]

When and how to invalidate the cache is of course also an important topic. Wrt. this topic I'm currently also not sure how often the static site would require rebuilding to refresh content, but I'll give an update once I know more. The question is whether it would be ok as a workflow if every cms update would require to rebuild the static site (eventually).

[jstet]

I wrote somethig regarding your last point in this hidden issue: https://github.com/CorrelAid/correlaid_website/issues/40

We could deploy a live preview that reflects changes in the cms and then build the page for production periodically, maybe once an hour or day, but allow for manual triggers.

not sure how easily we can switch adapters/use two at the same time for different deployments though

[KonradUdoHannes]

The only remaining linked issue is #182, which was moved to the backlog. As such this issue is considered done and the remaining linked issues remains open independently.