Static Generation

[GammaGames]

This is more of a question about implementation for a site. I saw some discussion on the main website repo (https://github.com/godotengine/godot-website/issues/82) and thought about how a similar approach could be used here.

Why not host the repository as a repo and generate the asset library api and site as a bunch of static pages using Sphinx or similar? I can think of several other package managers that do the same (alpine and winget).

I can see some pros:

• If assets were submitted as manifests (either yaml or json) they could be easily created or edited as pull requests submitted to the repository
• Assets would have a public history and could possibly include an automatic changelog based on pull request message bodies
• It could allow community members to continue to update assets if a developer stops maintaining it
• It could allow for the possibility of community-maintained asset repositories. Developers could host beta versions of their assets or completely private internal addons that would be easily accessible by users
• Could be hosted statically
• Would not require storing user information

There are a few cons of course:

• Would require a rewrite of the current asset library (pretty big undertaking)
• Would lock users into GitHub unless some cross-site merging system was created
• Anyone could update any asset, so it would require maintainers to validate asset updates

[Jummit]

You could also change the version in one commit, instead of changing it in the repo and in the asset.

[Calinou]

I did a proof of concept of this last year, but I didn't continue it as there would be no way to leave reviews. This is why I ended up working on a Laravel-based platform (which does support reviews).

[GammaGames]

Reviews and similar features would require an external service that would need to store user information anyway, so I don't think that should be implemented in a generated site and should be served dynamically. I do think simplifying the base asset library asset management is a potential idea worth exploring, though. @Calinou I would like to take a look at your proof of concept but it looks like the repo might be private.

Another pro could be that the site could be generated automatically when request is merged into the repo.

[Calinou]

@GammaGames Fixed, the repository is now public. I intended to delete it at first, but I'll keep it around if people want to take a look.

I'm not sure about making an hybrid system, as you may end up getting the worst of both worlds in this case. You have to manage a backend and deal with static site generator limitations at the same time. Keep in mind backend performance isn't an issue given our current setup and traffic. Moreover, we can also set up HTTP caching to decrease the load on the PHP interpreter and the database.

[bojidar-bg]

Good to see some discussion about it; there have been discussions of making the whole asset library be a GitHub repository with some json/yaml/ini files almost from the very start.

The consensus so far seems to be that it is easier to add features to dynamic sites, so a static site might have trouble with adding new functionality, effectively meaning that going full-static is a lock-in.

Pros of going static:

• Simpler and more transparent asset management; could even go full wild west, and read asset details from their repos directly. (Since it is a static site, such reading of repositories can happen more rarely, unlike when reading those repositories on every request.)
• Can be hosted on a CDN; also can be hosted more cheaply than a dynamic site. Can also be hosted on something like IPFS/IPNS.
• Makes it easier for people to create their own curated lists of assets, therefore decentralizing the system.
• Makes it easier to not track people. (Not that we currently do.)
• We get to throw away a few thousand lines of code (esp the part dealing with asset edits.)

Cons of going static:

• We get to throw away a few thousand lines of code
• Implementing reviews, ratings, or tracking downloads count becomes impossible without workarounds. Note that curated lists of assets can be argued to be better than reviews/ratings, and tracking download counts can be argued to be unethical.
• We have to tie the asset submission to a third party platform such as GitHub. Such a platform might track more about the submitters than we currently do.
• There are some challenges to be overcome in the way of searching assets.
• (Potentially a biggie) The asset library API for Godot 2.x - 3.x has to remain available at a fixed location and with a fixed interface. We can't teach older versions of Godot to search assets by grepping a large JSON file or something like that. Ideally, such an API would remain available until 4.1 or even 5.0.

[GammaGames]

Implementing reviews, ratings, or tracking downloads count becomes impossible without workarounds.

I do think this could be an issue, but I also think that authentication, reviews, usage, etc could be implemented with micro services that the static site (infused with some JS magic) could still interact with. I think overall it would probably save money and help break areas into more specialized components that wouldn’t need to be be so tightly coupled.

[bojidar-bg]

infused with some JS magic

Just a reminder that if one wants to display the reviews directly in the editor, they would either need an API gateway that interacts with those services, or a reimplementation of the JS magic. In the latter case, the C++ version of the asset library might become coupled with the microservices handling reviews/ratings, and that is not good. (And in the former case, why not go directly for a Vue webapp that uses the same backend?)