[RFC] Use SSG for frontend

jameshadfield commented 5 months ago

CSR, SSG, SSG, ...

This section is mostly background

The current site is entirely using client side rendering (CSR) - we send the client a minimal HTML page which then loads JS which then paints the page. See #810 for more details.

Pre-rendering the page as HTML (or as much of the page reasonably possible) at build time is typically called static-site-generation (SSG), but it's also referred to as a special case of server-side-rendering (SSR). Next.js mostly refers to it as SSR, but here i'll use SSG to differentiate from on-request SSR. The client still receives JS (essentially the same as in CSR) and once that's loaded that ~replaces the initial HTML ("hydration").

Per-request SSR (often just called SSR) is similar but the rendering to HTML happens on the server for each request. The HTML is sent to the client and hydrated. In theory you can cache a previous rendered version and use that as required, but it's unclear how much control/clarity Next.js provides here.

Both have advantages in certain situations such as improved SEO, faster time-to-first-paint, ability to work without JS. There are disadvantages which I'll expand on below. None of these advantages are all that compelling to me in the context of Nextstrain, but if you could have them for free then I'd take them. We used SSG with Gatsby, but it was broken (and no-one noticed?); see below for more on this.

Note: None of this is related to React Server Components (RSC). Next.js supports them. We don't use them. We can discuss elsewhere if people want to use them. RSC needs SSR, but SSR doesn't need RSC.

This PR

Implements SSG for 3 pages: the splash page, the /pathogens page and the /team page. One per-request SSR page is included, /example-ssr-on-request. The code is proof-of-principle, largely to explore whether we should be heading in this direction.

~To test changes you probably want to run with JS disabled or by throttling the connection to a slow speed so you can see any changes as the page hydrates. Logging in is useful as you will see the login button flash. Update: Can't login on review apps! For the time being you can compare with next.nextstrain.org (Next.js but entirely CSR) or nextstrain.org (Gatsby).~

Testing changes is easiest by running the server locally and comparing pages modified in this PR with those that aren't. Gatsby is no longer in use on any of our servers, but you could use a worktree to see a side-by-side comparison if desired.

Costs

The main costs in my eyes are the complexity / increased barrier to entry for developers who now need to be aware that the code will run both on the server and on the client. #810 was mainly motivated by reducing development cost, and I'm hesitant to add another layer we have to be cognisant of. If the rendering is different (server vs client) then it often results in a nasty flash as the page hydrates and the layout changes. Next.js (in dev mode) seems to highlight most hydration errors which is nice. A very similar and related case is ubiquitous on the web where the "login" icon will be shown initially and then it'll change to your name, behaviour which exists in this PR too. The Perils of Hydration goes into more detail and has many examples.

There are also other costs for SSR beyond those for SSG such as increased server load and less isolation between the server and client.

We were using SSG on Gatsby, right?

Yes. But it was rarely the same as the hydrated version. I don't think we really noticed? Below shows a series of screenshots, the far left is the Gatsby page with JS disabled (i.e. before it's hydrated), the middle is the Next.js page before it's hydrated (running from this PR), and the right page is the hydrated page (Next.js, but Gatsby should be identical).

Main splash page. Gatsby SSG is missing everything beyond the header.

/team worked well on Gatsby

/pathogens. SSG shows us the (animated) spinner because the resource listing data is fetched client-side by an API request. Note that there was a subtle bug in Gatsby where the SSG version didn't load the avatars in the footer and they'd flash in when hydrated; not a huge deal but an example of how these things happened and we didn't realise. This bug was flagged up by Next.js and fixed in this PR by 3cfb23925b82cbfba0899bf899d5b1f4b8aed730. Finally, If we used SSR we could skip the API request and load the page with the listing shown, but at a cost.

What to do?

I'm pleasantly surprised how nice the Next.js SSG experience is. I'm inclined to use it, but wanted to gauge opinions from others first. Beyond fixing hydration bugs, switching from CSR to SSG is a trivial change in each file within ./static-site/pages:

- import dynamic from 'next/dynamic'
- const Index = dynamic(() => import("../src/pages/index"), {ssr: false})
+ import Index from "../src/pages/index";

I don't think SSR is the right move, but I included a commit here to show how we could do it (c98760c350a91a0c5641a6e676fc972c9f1932bb is a more thorough way to share data between the node processes). A nice aspect of next.js' design is that we could use SSR for a select set of pages (e.g. /whoami) if we really wanted to, and even do so whilst maintaining the static-site/pages/* based routing.

tsibley commented 4 months ago

Thanks for this prototyping, @jameshadfield.

I think at the very least we should be doing some form of server-side rendering: either ahead-of-time (SSG) or on-request (traditional SSR) or both. The visitor experience just seems nicer (faster page loads, less initial render replaced by a big update a second later) at nearly no development cost. For some pages, I continue to think on-request SSR (over CSR) would make both our development experience nicer (avoid APIs we don't need) and the visitor experience (immediate paint, not delayed by further API requests).

I'm curious, why did you choose to stick with the older Next.js "pages router" instead of it's newer "app router"? I'd generally think to start with the latter…

jameshadfield commented 4 months ago

Thanks for responding!

I think at the very least we should be doing some form of server-side rendering: either ahead-of-time (SSG) or on-request (traditional SSR) or both.

SSG is (as this PR shows) straightforward and the current design of the site wouldn't immediately benefit from SSR so it seems there's agreement to do the small amount of work necessary to have SSG in the short term.

For some pages, I continue to think on-request SSR (over CSR) would make both our development experience nicer (avoid APIs we don't need) and the visitor experience (immediate paint, not delayed by further API requests).

Tom & I talked about this a week or so ago. Under the current design where all data is obtained through APIs then I still don't think SSR is worth it. Pros and cons to this design. Under a different design where you replace APIs with accessing the data at render time on the server side then of course SSR is needed (by definition). Trade offs.

I'm curious, why did you choose to stick with the older Next.js "pages router" instead of it's newer "app router"? I'd generally think to start with the latter…

I felt it was much closer in design to the Gatsby design and wanted to make a PR with as little changes as possible (both conceptual and number of lines). Nothing stopping future PR switching, either all at once or gradually.

nextstrain / nextstrain.org