Migrate the website to a static site generator

[willnationsdev]

cc @Calinou due to his related comment on Reddit

In order to make it easier for the community to directly contribute to Godot's website as needed, rather than access being limited to those with website credentials for October, perhaps we could migrate the website to a static site generator?

[mrzapp]

@willnationsdev It does seem like a more sustainable approach.

I think migrating templates would be easy enough, it's the content that could be bothersome, as static site generators tend to rely heavily on every page more or less just being a text document, so the structure might need some manual labour to correct.

I'm currently looking into upgrading the site to PHP7, after which I'll be a lot more knowledgeable about the challenges. If I end up with the impression that it would be feasible to do, I'll be glad to take on the project.

Does @Calinou, @reduz and @akien-mga have opinions on this?

[willnationsdev]

I have not brought the idea up with anyone specifically. And I'm not entirely sure what scale of work would be involved in-depth.

[mrzapp]

@akien-mga @Calinou After rummaging through the code and content, I am quite confident that it could be a static site, GitHub Pages with default Jekyll would even suffice.

The asset library might even be eligible too.

[Calinou]

I don't think we plan on using GitHub Pages for hosting, so we could go for Hugo and deploy the HTML ourselves on TuxFamily. I'd favor Hugo over Jekyll for two reasons:

• It's much faster. This matters because the number on articles on the website will only grow as time goes on.
• Its community seems to be more active than Jekyll's nowadays, so it's easier to find up-to-date solutions to problems.

Regarding the asset library, I started an Hugo-powered prototype but after discussing with @bojidar-bg, we were starting to notice limitations that can't be overcome well with a static site generator (like tracking download counts or reviews, if planned). This is why I started another prototype using Laravel, which is a much more complex task but it'd prevent us from running into limitations later on. I might be able to dedicate a significant amount of time to this task if my university agrees on it :slightly_smiling_face:

[willnationsdev]

Rather than Laravel, I would recommend Slim as it is just a lot simpler to work with. Would make it easier for people to get involved and, for a smaller application like this, will be easier to maintain overall imo. Thoughts?

[Calinou]

@willnationsdev The current asset library backend uses Slim. I think it's a bad idea for most sizeable applications:

• Slim isn't as well-maintained as other popular PHP frameworks. Its community is quite small, so it's difficult to find solutions to problems.
• The performance overhead incurred by the use of a framework isn't an issue in our case, due to the relatively low amount of traffic. (Still, frameworks make it easy to apply optimizations such as caching.)
• Most professional developers use a full-ledged framework nowadays. Using one of these frameworks makes it easier to turn them into contributors.
• Your code will look different from most other backends out there, which means contributors will have to adapt to your style and paradigms.
  • This becomes even more true as you need to scale up the application, as there's no "standard" way to do so.

[willnationsdev]

@Calinou good points all around. Sounds good then.

[mrzapp]

@willnationsdev @Calinou I have another idea that lets us have both. I suggest, given the need for future expansion, that we split this task into a public API service and a decoupled static site.

This approach has at least 2 major advantages:

1. A solid REST API would allow for tighter future integration with the Godot engine itself, as the client UI could be created with Godot widgets
2. We get to divide the work between server and client, making the process a lot leaner

If we create a specification of needed API endpoints first, then the frontend can even be developed against a dummy API, so one team doesn't have to wait for the other during development.

I am equally equipped for both tasks myself, I make a living doing webdev, so I'll gladly take on whichever is the most needed.

[willnationsdev]

@mrzapp This sounds like a really good approach. I likewise make a living with webdev, though I'm not quite as experienced as others here may be (only been in the professional world for about 3 years).

[Calinou]

A solid REST API would allow for tighter future integration with the Godot engine itself, as the client UI could be created with Godot widgets

Don't we already have this, or am I misunderstanding something? https://github.com/godotengine/godot-asset-library/blob/master/API.md

[mrzapp]

@Calinou it seems we do. Then don't we just need to expand this API to support the download count, reviews and other needed features and write a decoupled static frontend to present its content? The entire asset library frontend could literally just be 2 page templates (1 for the overview and 1 for individual assets) of the main Godot website.

[Calinou]

@mrzapp By "static frontend", do you mean setting up a single-page application using a static site generator? That can be done, but if we're going to rewrite the backend, we can do that directly in the backend's codebase (Laravel plays well with Vue.js, for instance).

[mrzapp]

@Calinou sort of. I meant that instead of the asset library having a frontend built in, it would only be a web service with a REST API, and the web frontend would be a page template of the main Godot website.

I'm trying to advocate a "separation of concerns" approach first and foremost. It provides a kind of flexibility that a joint effort like this benefits a lot from:

• Developing the front- and backend independently of one another would allow us to start the work on the new static Godot website right away, including the asset library page, using the existing asset library backend as a web service.
• Focusing on the new asset library backend exclusively as a web service would keep the codebase lean and readable.
• Changes to the asset library backend would not affect the frontend at all, or at least minimally, which would maximise the overall uptime and reduce the deployment complexity.
• Likewise, should the need for a new frontend arise, the backend would remain untouched.
• Making the asset library frontend a part of the main Godot website provides a smoother user experience, as they would be sharing the same codebase. Currently there's a stark UI/UX difference between the two.
• The development responsibilities would be divided sensibly, meaning that we won't be making bottlenecks out of individual developers, should they go on holiday, be too busy at work or get a flu.

Am I selling this yet? 😅

[Calinou]

@mrzapp Sounds good to me :slightly_smiling_face:

[mrzapp]

@Calinou I've knocked together a prototype with Docker using the existing asset library as an API service, if that will help sell the concept just a tad more.

@willnationsdev Would you be able to work with something like this? What's your area in webdev?

[willnationsdev]

@mrzapp I've been working with lando recently in the last couple weeks, so I have some experience there, and I've experimented a tad with Laravel, if someone chooses to use that for the backend API. It was confusing, but I'm sure I'll manage to learn it more if I have an opportunity to contribute to the project.

[bojidar-bg]

Just to chime in: I think most of the asset library might be moved to a static website. The only things that need to be dynamic might be done using JavaScript (for comments and reviews) or simple PHP scripts (for downloads -- the download link would just lead to our script, which then redirects or streams the response). Things such as assets can be kept as static JSON files (already tested by @Calinou, AFAIR), and maybe mirrored in a database for searches. Again, the script which does the search could be quite simple. Finally, there would have to be a compatibility layer, which would be a set of scripts which the old versions of the engine can still use.

[jcs224]

@willnationsdev @mrzapp @Calinou Bumping this as I just ran into a problem with trying to run the OctoberCMS site on Windows. Looks like the workflow for working on the site in a local dev environment favors a UNIX environment (.sh scripts and whatnot). While I could (and probably will) work out the issue, I would prefer separating the website from the content. Static sites make this very easy.

I recently converted my personal site from WordPress to Jigsaw and found it much easier to make updates because I didn't have to create an entire environment for running PHP, a database, etc. Just the static site generator.

I agree that Hugo is probably the best option if we go this route. It is very popular and easy to get going.

In fact, we might not even have to ditch OctoberCMS for managing content. Just browsing it looks like there is REST API plugin that Hugo could consume to keep the workflow the same for content editors. When someone develops locally, when they build, it would just pull from the REST API in production so they would get the latest content without having to set up PHP/MariaDB environment on their machine.

[willnationsdev]

Not sure if this is at all relevant to this project, but it seems that Microsoft will stop creating Windows-compatible versions of PHP, starting with 8.0. I'm guessing Godot's stuff runs on Linux, but it does make contributing harder for folks on Windows machines if they literally can't run PHP to develop the site in tandem with everyone else in the community. At least, if I'm understanding this correctly.

[jcs224]

@willnationsdev I don't think that will affect things practically too much. PHP runs 80% of the web so someone will carry on compiling for Windows, or use WSL. It would especially affect me because I'm a Laravel dev professionally but also like to game and make music, so Windows it is for now.

Anyway, I think we should work to make it as easy to contribute to the website as it is to contribute to the game engine itself.

[Calinou]

I'm in favor of migrating to Hugo, but this is a large task that needs to be approved by @akien-mga first. Last time I heard, he would rather have me focus on higher-priority tasks :slightly_smiling_face:

@willnationsdev There will still be PHP for Windows, it just won't be maintained by Microsoft anymore.

[mrzapp]

@jcs224 you might want to look into running your webdev stuff in WSL, Docker or a virtual machine, if you're stuck with Windows. The web runs Linux, so there is little incentive to make web apps Windows compatible.

Either that, or start making music and gaming in Linux, which is what I'd always recommend ;)

[jcs224]

@Calinou sounds good, I understand. I would be willing to take a crack at this if there is consensus that it is a direction we want to move towards. Or maybe I'll just do a proof-of-concept anyway 🙂

@mrzapp WSL would probably be the way to go, it just hasn't worked well for me in the past. Guess I can try it again. Easier said than done moving to Linux. Games are getting pretty easy to run on Linux, but support for audio drivers is abysmal, especially for newer laptops.

My only objective here is to remove friction. I'll try to make the current system work with Windows and make a PR to the docs if I can get it going.

[celioreyes]

Hello 👋

Is the hugo port of the website on pause? I was looking through some of the website issues and this one seems right up my alley due to working on a static site workflow at my day job for a few years. I'd love to contribute to moving the website over to hugo if others are working on higher priority tasks.

[Calinou]

@celioreyes We haven't started any porting work yet. I would recommend getting formal approval before starting any work, as you wouldn't want to port the website and never get it merged in the end.

[celioreyes]

@Calinou yeah that makes sense! I'll make keep an eye out to see where this lands 🙂

Are other issues up for grabs or do most of them require an approval? (first time contributor and want to make sure I'm not stepping on anyone's toes)

[Calinou]

Are other issues up for grabs or do most of them require an approval? (first time contributor and want to make sure I'm not stepping on anyone's toes)

There's not a whole lot of work to do on the website these days. That said, contributing to the documentation is always welcome. See the main documentation and class reference contribution guidelines.

[Calinou]

Closing, as we discussed this with reduz and others and this won't be done to ensure the website remains easy to add blog posts to. Using a CMS instead of a static site genreator also makes it easier to keep certain blog posts private until they are published (which may be required for grant announcements and the like).