Website migration to Vercel

[ovflowd]

Hey, I'm opening this issue to keep publicly tracking the conversation we have with a currently ongoing effort to evaluate a possible migration of the https://nodejs.org website toward Vercel infrastructure.

This consideration initially started because we're adopting Next.js (full SCG) with https://github.com/nodejs/nodejs.org/pull/4991 and the fact that by adopting Vercel's infra, we would have a set of benefits.

It is essential to mention that if we decide not to use Vercel infra, we might still want to consider having our Next.js installation run with SSR (Server-Side Rendering) on our Infrastructure maintained by the @nodejs/build WG.

What is this proposal about, and why?

On why adopting Next.js SSR/ISR

• Incremental builds that allow us to quickly build on our production and staging infrastructure with seconds between each deployment
• Internationalisation features that increase our accessibility to an international audience. Such as automatic language detection and redirection (Accept-Language), hreflang, and other features.
• Built-in Middleware that is highly customizable and would allow us to offload some of the Rewriting Rules we use on NGINX that are harder to maintain (require special access to the server, more prone to error, require specialized knowledge on NGINX).
  • This would allow us to have better i18n routing/redirection, better 404 redirection/routing
  • This would allow us to ditch the /pages/{twoLetter} directory format of having one repeated page for each locale and adopt a similar design from https://github.com/nodejs/nodejs.dev, where we have a single file for a page and multiple files for content. (Example: here and here).
• Caching of Assets, Pages, and API requests.
• It would allow us to introduce some of the Microsites, such as the Google Calendar microsite, instead of using the default Google Calendar Share page
• It would allow us to introduce API requests to lazy-load/offload rendering of specific components that are performance hungry, like the Previous Releases page that loads a huge JSON on every request.
• Introduction of Search (with Elasticlunr or alternatives) with indexing, server-side processing, and caching. Which would allow us to search for content on the website. (Gatsby supports that on SCG, but with the caveats that the client does the processing "statically".
• And more features that Next.js support.

On why adopting Vercel infra

• Easier management of deployments and a more seamless experience for live previews, staging branches, rollback, metric management, load balancers, and better auditing of deployments
• Not a single point of failure, as it is not hosted on a single server
• No monetary costs
• Less requirement of dedicated knowledge of our current build system infrastructure and special access and unique understanding of our legacy systems

What is this proposal not about

• Only the rules/pages handled by the nodejs.org repository are what we're considering offloading to Vercel. This means only the NGINX rules that currently point to the statically generated files from the nodejs.org repository, as pointed out here
  • In practical terms, a proxy (rewrite rule) would be added in place to forward the requests to the Vercel-hosted App (as a reverse proxy, forwarding also headers, query parameters, et cetera)
• All dist, downloads, releases, dist.json, generated metadata, other subdirectories, and microsites will remain on our current Build Infrastructure with the same NGINX rules.

What was talked about so far

• We had initial discussions with Vercel's Staff
• Meetings were held off with the Website WG and some people from the TSC and Build WG together with some people of the TSC and together with the Foundation Staff (@joesepi, @richardlau, @Trott, and others)
• In-Slack communication has happened with several people from the Foundation, Vercel, and Node.js
• This issue was finally created to publicly "audit" that this whole proposal is happening
• I believe further talks with the Build WG, Release WG, and the TSC will be done, including more discussions with the Foundation.

Next Steps

To be documented

[Trott]

• Meetings were held off with the Website WG and some people from the TSC and Build WG together with some people of the TSC and together with the Foundation Staff (@joesepi, @richardlau, @Trott, and others)

The others were @rginn and @kyliewd.

[mcollina]

I'm absolutely in favor of adopting Vercel to improve the dx and have staging branches etc. However our website is mostly static information: it seems a waste of energy to render things all the time.

My understanding of this proposal is that we would deploy:

1. Cloudflare CDN on top
2. Nginx on our single server: a. to serve the release downloads. b. to proxy to Vercel.

This does not improve the reliability of our infrastructure because it's still passing through our single nginx server. It will instead increase the load on our single server and worsen the response latency for our end users. I'm not going to block this for that anyway.

I think that the proposed architecture is less reliable and not something we want to have. Happy to help designing something better with the tech at play (e.g. completely bypass our nginx and proxy vercel from cloudflare).

[ovflowd]

Thanks for the feedback, @mcollina! I also feel like using Cloudflare directly could be an interesting solution. Maybe, through their Workers feature or something (as it allows path matching)

Anyhow, just wanted to make it clear I just gave an example, of how we could achieve using Vercel just for the website without getting rid of the current infra for downloads/releases.

I do believe using SSR will come at handy for all the features we want to introduce. It also improves build, reduces memory usage during build time and allows us to do other things that right now we've done some not ideal approaches on the initial PR for Next.js.

[leerob]

Hey folks! Happy to answer questions about Vercel.

I'm absolutely in favor of adopting Vercel to improve the dx and have staging branches etc. However our website is mostly static information: it seems a waste of energy to render things all the time.

What do you mean by a waste of energy? Is this for Vercel? If things are static, those generated assets are deployed to our Edge Network (CDN). You wouldn't need to use another CDN on top.

I do believe using SSR will come at handy for all the features we want to introduce. It also improves build, reduces memory usage during build time and allows us to do other things that right now we've done some not ideal approaches on the initial PR for Next.js.

This is one of the nice things about Vercel – you can choose static or dynamic, on a per-page basis, without needing to change infrastructure. The Node.js site currently has a lot of pages (I think over 1,000?). Plus you have internationalization, so multiply 1000 times the number of languages. You'll likely want to defer the generation of some of the infrequently used pages outside of the build (to prevent really, really long builds).

As mentioned above, I've approved an open-source sponsorship to provide a free Vercel team for the Node.js site 😄

[mcollina]

What do you mean by a waste of energy? Is this for Vercel? If things are static, those generated assets are deployed to our Edge Network (CDN). You wouldn't need to use another CDN on top.

The content is mostly static, but this proposal is about making some of them dynamic to deal with a few rough edges of the current solution.

---

Why we need a CDN/NGINX on top

Most of the complexity we need to deal with comes from technical debt.

We are currently serving all Node.js binaries from the same infrastructure of our website. We cannot easily change those URLs because a lot of other tools everybody uses depends on them. Latest stats put this traffic at around 4.5million daily downloads during weekdays. (Cloudflare+NGINX allows us to serve all of this from a single server).

As you would expect from a project of this size, very few people can push binaries. On the other hand, we want a contributor friendly website, and let people publish changes to our website from GitHub.

I would personally feel uncomfortable deploying the binary downloads as part of our website.

[ovflowd]

@mcollina I don't want to sound like I'm repeating myself (So I apologize if it sounds repetitive), but again, we don't want to (for now, it's not even part of the plans) change where the binaries and downloads and archives are. Just the live website 😅

(Note that we didn't mention at any moment that we should get rid of Cloudflare or NGINX)

[mcollina]

@ovflowd I was responding to @leerob that said:

You wouldn't need to use another CDN on top.

However, this is exactly what we are planning to do.

[ovflowd]

Gotcha, ty for clarifying, and sorry for the confusion 👀

[mhdawson]

@nodejs/build to make sure build team is aware.

[leerob]

Ah, apologies @mcollina I was misunderstanding how you wanted to serve those binaries. Agreed that putting Cloudflare in front of Vercel here would make sense!

[mhdawson]

Teasing apart the downloads from the website itself is a good goal, but as Matteo mentioned this part does not get us many of the benefits:

b. to proxy to Vercel.

I guess the challenge is that the download/sites are tightly coupled in terms of existing URLs and we can't use dns to have traffic directed to different places for the downloads/site content?

If that is impossible then possibly using CloudFlare as the proxy to Vercel/our existing server for downloads might be possible (that might possibly be what was suggested above). We mighte even be able to set it up so that we do both in parallel to test out where test.nodejs.org splits requests across the two locations while nodejs.org continues to serve both from the existing server?

[ovflowd]

I guess the challenge is that the download/sites are tightly coupled in terms of existing URLs and we can't use dns to have traffic directed to different places for the downloads/site content?

Not really, if we do it through NGINX it's pretty easy, but it doesn't reduce the load on our servers. It would be a temporary solution.

Afaik, Cloudflare supports path-based proxying tho. It is something that can be done through Cloudflare Workers, as I mentioned before...

We could also, of course, make a Cloudflare rule to redirect the downloads/binaries path to get.nodejs.org or something of sort.

[ovflowd]

@mhdawson quick-question: do we use a free Cloudflare license?

[targos]

We have a free Business plan

[ovflowd]

We have a free Business plan

Sponsored? Cool, I suppose then all usage pricing goes to 0, right? Because of https://developers.cloudflare.com/workers/platform/pricing/

[targos]

It also says that we have free access to Workers (don't know if it's unlimited), but can't we just use Page Rules to proxy based on the URL?

[ovflowd]

It also says that we have free access to Workers (don't know if it's unlimited), but can't we just use Page Rules to proxy based on the URL?

Page Rules is also handy and can do the same, indeed.

[mcollina]

I think we would need to add page rules to route the website to Vercel and the downloads to nginx.

[ovflowd]

Sounds good to me :)

[mhdawson]

From the discussion it sounds like:

We might even be able to set it up so that we do both in parallel to test out where test.nodejs.org splits requests across the two locations while nodejs.org continues to serve both from the existing server?

might be possible? @ovflowd do you think it would be based on what you undertstand about cloudflare?

[ovflowd]

Yup we can make it happen :)

[mhdawson]

@ovflowd good to hear. That seems like a way to show it working in advance.

There is still the question of getting agreement from @nodejs/build and @nodejs/tsc that depending on an external site to host the website is something we want to do.

[tniessen]

This is one of the nice things about Vercel – you can choose static or dynamic, on a per-page basis, without needing to change infrastructure. The Node.js site currently has a lot of pages (I think over 1,000?). Plus you have internationalization, so multiply 1000 times the number of languages. You'll likely want to defer the generation of some of the infrequently used pages outside of the build (to prevent really, really long builds).

I think the vast majority of these pages are blog posts and API docs, neither of which are internationalized AFAIK.

[ovflowd]

@tniessen the multiplication is still valid because even if the content itself is not translated, the pages are and served on all the available languages.

You can simply run the build command of current website and see the thousands of html pages that it outputs 😅 (and that is without incorporating API) so imagine adding API on too.

That is basically because the static model has no routing, so any of those pages must be statically available on all those languages, even if the actual translated part is just the headers or some common i18n elements.

[tniessen]

Ah, I see. npm run build takes less than five seconds and is probably good enough for many local scenarios, but npm run build:deploy takes ~40 seconds locally.

As far as I can tell, metalsmith neither caches build outputs nor parallelizes the build. If build times are a concern, if we replace metalsmith by some other build process, that would hopefully be addressed.

(and that is without incorporating API) so imagine adding API on too.

Why? Aren't these just static files with no internationalization?

[ovflowd]

Still, metalsmith is blazing fast (mostly due to its simplicity compared to a full blown React Framework, but still super props to Metalsmith!!), and Next.js full SCG build takes 2-3 minutes in comparison with Metalsmith. What we want to address here is not build time, but the flexibility on addressing things such as i18n.

On the current way, yes API docs are statically built and have no incorporation, I'd ask you to head over https://nodejs.dev/api/ and see some of the plans of how we plan to incorporate the API docs to the actual website 🙃

[tniessen]

Ah, thanks for explaining. I can see how switching to a slower build system while also vastly increasing the number of generated resources could be problematic.

[ovflowd]

Indeed, here actually ISR/SSR kinda becomes a "must" because the build system is slower, for production at least. But of course that's just one of the reasons of adopting ISR/SSR...

But that's kinda the tradeoff from a simpler framework to another that does a plethora of things, for better or for worse.

[nschonni]

The slow build of the main site is only an issue for onboarding the docs that isn't a current concern with the existing website. I'm still not sure that splitting that off to docs.nodejs.org and going full Vercel (or something like that) isn't a better approach. Since I'm not doing the actual work on the website anymore, I'm not going to push that too hard.

[ovflowd]

@nschonni could you rephrase your comment? I didn't get a single thing 😅 what you mean with splitting the docs and the onboarding part?

[nschonni]

Like a few of the threads above, just repeating that the current site content is static and faster with the current build time compilation.

What I'm suggesting is to look at the parts where the new hosting is going to solve problems over the static content (API docs, calendar, Learn), while leaving the non-interactive pages with Metalsmith. This ISR/SSR approach could be selectively leveraged as microsoties like you mention above, without changing the current nodejs.org infrastructure outside of additional redirects for some old paths if the API docs would be served at something like docs.nodejs.org backed by Vercel or something else.

PS: Just finished reading https://www.zachleat.com/web/react-criticism/ which I guess has been floating around tech discussions this week and may be part of the concerns about moving a static site to a front end framework

[ovflowd]

while leaving the non-interactive pages with Metalsmith

Having a mixed framework using Handlebars and other JSX would be a nightmare. We would need to support two complete stacks (MEAN/JAM) with entirely different underlying mechanisms... I don't see any benefit from having this 😅 (It would require coding compatibility layers so we can use data such as translations between both of them; it would very possibly require doubling the templates, ensuring both support the same styling... It is just a massive hassle).

This ISR/SSR approach could be selectively leveraged as microsoties like you mention above, without changing the current nodejs.org infrastructure outside of additional redirects for some old paths if the API docs would be served at something like docs.nodejs.org backed by Vercel or something else.

I'm not much of a fan of fragmenting the website; I honestly believe that if fragmenting into smaller packages or "microsites" is done purely for build performance reasons, that is too much of an inconvenience.

The build difference between both is insignificant by nowadays standards. Being picky because of extra 1-3 minutes of production build time on full SCG is... at best, petty since this is just a website that gets updated occasionally.

Having SSR/ISR globally available has its benefits. There are many at hand, and my main takeaway for SSR is not about the performance gains but the server-side features we get from this (From routing, internationalization, APIs, Caching, and others).

PS: Just finished reading zachleat.com/web/react-criticism which I guess has been floating around tech discussions this week and may be part of the concerns about moving a static site to a front end framework

Instead, I'd leave this topic out of this Issue, as the point here is about adopting SSR (and Vercel) for the upcoming Next.js migration, and not what React's flaws are. (Note that the blog post's author is opinionated by being one of the 11ty maintainers).

(Also, The Node.js website is not a product; most of the concerns in that blog post don't apply here (they're still valid but do not apply here).

Yes, many web frameworks/static content generators are out there, and they all have issues, drawbacks, and shortcomings. Hence I believe we should focus on the topic at hand, whether adopting Vercel infra benefits us.

Finally, one of the main takeaways about adopting Next.js is primarily due to Developer Experience. We want something familiar to newcomers and to use widely adopted technologies with comprehensive documentation and widely known (TL;DR, we want something newcomer friendly). We also want to use something that can be supported for generations ahead.

[mcollina]

I'm +1 in using Next.js or React for the website, even if it increase the build time. It greatly improves the maintainability of our website, lowering the barrier to entry for new collaborators.

I'm not sold in using SSR features given that the website is (right now) mostly static data.

[ovflowd]

While a good part of the content is relatively static, many aspects are not; of course, it all depends on what static would be in the long run. If the same content needs to be updated even once because of its change, then it is not genuinely static. A blog post is static.

A previous release table is not, but it does not often change over time. So quite static. It can be leveraged during build time.

Now, internationalization is not static, it is pretty much request-based. As the router needs to evaluate HTTP headers. So ok, not static content at all. But well, Internationalisation applies to all pages. So, whereas the page's content or a vast part of the content is static, a few surrounding mechanisms are not.

And here is where SSR comes into play, not necessarily for its actual server-side rendering aspect (since Next.js static builds already include initial renders), but the "server processing" side. This allows us to add a website search feature with GraphQL, something more straightforward, or even Firebase/Elastic.

It allows better Internationalisation with the detection of preferred user languages and locales; it will enable us to reduce by far the building footprint as we do not need to generate all pages at once nor the same page for all different languages (even if the core content is in English; buttons, headers, footers, and other parts would still need to be translated).

It allows also better caching at page and resource levels. It will enable image optimization, font optimization, and pre-rendering of pages a user intends to open (Next.js feature), and the list goes on...

So for me, it is all about all the features for a UX and DX experience we can get by using Next.js's SSR. Or, more specifically, Next.js server (CLI/Runtime, whatever you call it).

Does that kind of makes sense for you, @mcollina?

[tniessen]

@ovflowd Is my understanding correct that we could maintain the ability to entirely drop vercel/SSR at any time and "only" lose features such as automatic internationalization based on request headers etc.? I don't think site-wide search, for example, is an integral feature.

[ovflowd]

@ovflowd Is my understanding correct that we could maintain the ability to entirely drop vercel/SSR at any time and "only" lose features such as automatic internationalization based on request headers etc.?

Yes. We can disable Vercel/SSR at any point, and the website would still work (perfectly) if we keep the current "two-letter" folder structure. (Which is fine for both SSR and whatnot), but we wanted in the future (ideally) to separate content from pages (and their respective layouts).

site-wide search, for example, is not an integral feature.

What do you mean is not an integral feature? We don't necessarily need Server-Side for a search feature; that's correct. But with Nextra (Next.js) 's search functionality, it would. The client-side search would require us to create data bundles (most likely JSONs) that can be lazily loaded, and then the client can filter from the JSON data set any match that matches the search.

Overall, I'd like to use SSR to have a better search that doesn't require the client to load a big (third-party json bundle) or process all the data, as it can be costly.

[tniessen]

but we wanted in the future (ideally) to separate content from pages (and their respective layouts).

I'm sorry, I'm not familiar with Next.js. Why does this separation require SSR?

What do you mean is not an integral feature? We don't necessarily need Server-Side for a search feature; that's correct. But with Nextra (Next.js) 's search functionality, it would. The client-side search would require us to create data bundles (most likely JSONs) that can be lazily loaded, and then the client can filter from the JSON data set any match that matches the search.

What I mean is that I think any existing search engine is doing a good job at indexing nodejs.org. For API docs in particular, client-side search using browser tools (within currently open documents) is often helpful enough. I guess what I should be asking is what you hope to gain by implementing this. Is there evidence that a custom site-wide search feature actually be more accurate or convenient?

[ovflowd]

I'm sorry, I'm not familiar with Next.js. Why does this separation require SSR?

Sorry, I misphrased things, I meant to say that the current "content structure" that nodejs.org adopts uses. But as soon as the initial Next.js migration gets merged we can update the Frontmatter and the directory structure to follow something where we have a content folder and pages folder. That is indeed unrelated to SSR.

[ovflowd]

What I mean is that I think any existing search engine is doing a good job at indexing nodejs.org.

That's not what I meant, with site "search" I mean a search box for people to search content within the website :) (That is a very requested feature)

Is there evidence that a custom site-wide search feature actually be more accurate or convenient?

I can't really say for hard evidence, besides of allowing easily users to find the content of the website + searching for methods, classes and modules on the API docs; I also see that most of the popular frameworks and programming languages have site-wide search 👀

[tniessen]

a search box for people to search content within the website :) (That is a very requested feature)

I see. I really did not expect that to be a popular feature, but maybe that's just my personal preference.

---

Thanks for the discussion during the TSC meeting @ovflowd. As @richardlau said during the meeting, not having to rely on our own infra for hosting content would be nice. But I tend to agree with @mcollina that we have more flexibility in hosting content if it is static (whether that's nginx, Vercel, Cloudflare pages, GitHub pages, AWS, or something else). I assume that incremental builds would solve the build time issue (if it really is an issue).

I assume that adding routing based on internationalization headers would be possible in most setups (except perhaps GitHub pages). Server-side search appears to be the only feature that would force us to commit to a particular technology and/or infrastructure.

[ovflowd]

I see. I really did not expect that to be a popular feature, but maybe that's just my personal preference.

And something we can debate; nothing is set in stone 👀 but I can see benefits for people to search content through the "Learn/Guides" and API... And maybe even find a blog post.

For example, on https://nodejs.dev, if I want to know how to write content to a file in Node.js, the search is already intelligent enough to give me some nice results (including from the API)

But I tend to agree with @mcollina that we have more flexibility in hosting content if it is static (whether that's Nginx, Vercel, Cloudflare pages, GitHub pages, AWS, or something else). I assume that incremental builds would solve the build time issue (if it is an issue).

For all purposes, even if we adopt SSR, the website must be able to serve the content without SSR. This means all features we get from SSR should be the "cherry on the top of the cake". We've failed if we depend on SSR to make the website work/build.

. I assume that incremental builds would solve the build time issue (if it really is an issue).

Even on Vercel, the whole thing is built statically, with incremental builds and SSR on the backbone allowing extra features to exist.

I assume that adding routing based on internationalization headers would be possible in most setups (except perhaps GitHub pages). Server-side search appears to be the only feature that would force us to commit to a particular technology and/or infrastructure.

We could even achieve that with pure NGINX, but here's where using Next.js as the server (behind Cloudflare) is excellent; all the routing can be done there. Also, my idea is that the Routing middleware should "work on top" of what we have. By no means do I want to rewrite the website so that it gets dependent on Vercel's/Next.js's Server-Side routes.

[mhdawson]

Discussed in TSC meeting last week so removing the TSC agenda tag. If there is a specific ask from the TSC feel free to add it back on.

[ovflowd]

@mhdawson I think from the TSC-side, we got a 👍, right? I believe we can forward that to the Foundation to proceed with the request, right?

[mcollina]

I would prefer to see a PoC of cloudflare/next/downloads before committing.

Anyhow, I think we would need support from Vercel for the documentation previews for the new Node.js docs, so I wouldn't object moving things forward with them.

[ovflowd]

We can do a PoC regardless of what we plan to do with Vercel, but we need access to Vercel first, right?

[leerob]

Let me know how I can help with the Vercel setup.

[tniessen]

Since the website is now based on Next.js (and since that likely won't change anytime soon), I am not against supporting SSR -- as long as it remains entirely optional and is not tied to any particular vendor. I don't see the need for APIs or anything fancy at the moment, aside maybe from a search feature if people really want that. If we need vendor-specific code, it should be optional (and as little as possible).

Assuming this is going to be vendor-independent and given that we already heavily rely on Cloudflare, could we just as well host it there to avoid further complicating our infrastructure setup? I am sure Vercel is an awesome service; I simply have no idea what exactly is being proposed and/or required here since I don't use Next.js.

[ovflowd]

Yeah, the idea is still have no vendor-based code (vendors being Vercel) inside the codebase and as less as possible framework-specific imports in the codebase. Right now we have just a handful, if I recall correctly.

But adopting Vercel as our Infrastructure, would lift a huge burden we have right now on the very small Build team. Not to mention many cool features, like like Git integration, Branch Previews, better caching and et cetera.

We definitely need to check how we integrate that on Cloudflare and we're working on some PoCs. But yes, for maintainability and future proofing the solution, the repository aims to be as much as vendor independent as possible.

[mhdawson]

@mhdawson I think from the TSC-side, we got a +1, right?

My take is that there were no objections to the direction in general of a hosted environment outside of a machine run by the build wg for the Website content. As mentioned by @mcollina a good next step is a PoC showing the current downloads and website being split up even if they are still hosted on the same machine. Once that was in place it should be easy to try out shifting the website to vercel through vercel while being able to fallback to the instance we have running on the original machine.

[ovflowd]

As mentioned by @mcollina a good next step is a PoC showing the current downloads and website being split up even if they are still hosted on the same machine. Once that was in place it should be easy to try out shifting the website to vercel through vercel while being able to fallback to the instance we have running on the original machine.

But this is already virtually happening. Both downloads and the regular website are being served by different nginx blocks. 🤔

But yeah, I think an actual feasible PoC would be to actually set up a testing nginx config with similar infra in a free/staging Vercel environment. We maybe could spin up a machine on our DigitalOcean account.