Reduce unnecessary requests

[Aeledfyr]

Currently the client requests the entire list of versions from Mojang on every page load, which adds about 100KiB to every page load. It may be more efficient to request the versions list once on the server side, and then send a more minimal list of versions to each client. Then, if the client opens the snapshots or archaic versions lists, the client could request the full list from Mojang's servers. This should reduce the loading time of the page, as well as prevent Chrome complaining about having >400 children of a single element in the snapshots tab.

The other large requests that are currently being sent are mainly from the icons for mods, which are much higher resolution than will be displayed. I don't know if there are smaller versions of the icons on the curseforge cdn, but that may be worth looking into.

I am currently minifying the svg icons, which are another thing we could improve - if we could reduce the number of requests for the icons, that could reduce loading times. Since most of the icons are svg, it is theoretically possible to put them all on a sprite sheet and then reference them individually, but that requires more work.

[Geometrically]

For the client requests -> Should we move those to the server on startup? So it is baked into the template and sent to the client instead of the client requesting the MC Versions from launcher meta?

For the mod icons -> Curse's CDN does allow you to query smaller versions, so I'll have to look into that. I think a good resolution would be 64x64. That would save bandwidth by a ton and allow icons to load a lot faster

As for a spritesheet, how would that reduce the load times? I'm not familiar with that kind of stuff but the SVGs are pretty lightweight anyways, iirc they are only about 20 KB for the entire images directory (excluding the donation PNGs)

[Aeledfyr]

The server should request and parse the versions list on startup (and probably every 12 hours / whenever we recheck curseforge), and then bake the main versions into the template. Baking the snapshots in would double the size of the page - the html for the snapshots list is 30KiB on it's own, so we should only bake the main versions, and trigger the version fetching if the snapshots list is opened. To make the second version fetching more efficient, we could have an api page that serves a json list of the current cached versions for snapshot/release/old, which would reduce the bandwidth for the client and server.

The mod icons solution looks good. Some mods don't have icons, so it may be a good idea to have a basic fallback image to insert when no icon was found. For an example, "Aroma1997Core" has no icon.

The icons directory was 156KiB, though I now have it down to 124KiB through minifying/compression. A sprite sheet wouldn't save that much bandwidth—only the overhead of http requests—but it would reduce the number of requests the browser has to make from ~20 down to about 1 for icons. Requests to a single server aren't generally made in parallel for HTTP/1, so more requests have a larger loading time overhead. I would also assume that having a single image file would also be more cache friendly for the browser, but I can't be certain about that.

[Geometrically]

For the snapshots could we do the templates client-side? Or maybe that isn't worth the size of the handlebars DOM on the client. I'm honestly not sure how to handle this without it being a loss situation. One easy fix is making the launcher meta requests async load, so it doesn't slow down load times. The user won't be clicking the latest filter anyways right when the page loads.

Otherwise, I don't think there's a good way to do it. Generating the versions client-side while does bill compute saves tons of bandwidth. Since it's already doing snapshots it might as well do versions.

I can make a missing icon soon. It is strange seeing the browser missing icon effect.

For the sprite sheets, how complicated would those be to use? Actix uses HTTP/2 so the requests should be in parallel. However it does come to a point where the optimization isn't essential. Currently on staging the majority of the response times are from immibis's server being in Europe, which slows down stuff quite a bit. In production we should have all static content stored on some sort of CDN which does caching already

[Aeledfyr]

We're already generating the DOM for the version lists entirely on the client side, so we should be able to keep the same code that we already have - to minimize bandwidth use and still reduce load times, we could host the versions list in a minimized JSON format on the server, and have the clients request it from there. The JSON size should be about 6KiB max, rather than the 120KiB of Mojang's official version.

The sprite sheets are easy to use, but are slightly harder to add icons to. I would recommend just using SVGs normally for development and prototyping, and then at some point go back through and replace them with references to a sprite-sheet. Even with HTTP/2, it should still speed things up, but if it isn't worth the effort, using minified SVGs is good enough.

I've implemented a sprite-sheet system in 2e900434ad2adeadeae6efbf2a95999b91a1db8a as an example, as well as the fix for thumbnails.

[Geometrically]

I see what you are saying, have an endpoint on the server for minecraft versions as the launchermeta contains tons of extra data. Sounds like a great idea, I can look into that tomorrow unless you want to do it.

Spritesheets seem awesome, however I do have some concerns! Is it one spritesheet per page? or per site? both would be wasteful, since some SVGs are reused throughout the site, causing waste on both the server's space and the client since it can't cache them. Per site would allow for caching, but not every page needs every SVG, so the first load for the client would be quite long. This would impact bandwidth, however page load times would decrease.

If you could address those concerns that would be great. If we do switch to them, like you said, SVGs would be used for dev and then moved into spritesheets for production.

[Aeledfyr]

On the versions list: filtering by 1.16 appears to be broken for some reason - I'm not certain why. The curseforge indexing setup doesn't currently handle snapshots, and including the snapshots listing on the sidebar likely has a performance impact, because it adds over 400 elements to the DOM. It may be more efficient and more useful to have categories like 1.16 snapshot, 1.16 prerelease, etc, as well as some of the special versions like 20w14infinite. Otherwise, finding mods on snapshots or prereleases would become tedious, as you would potentially have to select many different versions. Are there enough mods on the snapshots to justify selecting individual snapshot versions, rather than selecting 1.16 snapshot and sorting by the most recently updated? We could theoretically add a menu to input specific snapshot versions if that is required, but I doubt that enough people would search for mods on the snapshots to justify having a list of all previous snapshots loaded by default.

For spritesheets, it would be one spritesheet per icon set - so all of the category icons would be together, all of the mod info icons would be on one sprite sheet, and the icons used for other parts of the UI would be on another sprite sheet. This allows for each page to request the spritesheets that it needs without too many unused images and should work well with clients' caching. Since the groups of images would already all be requested at the same time, combining them into one spritesheet should just reduce the overhead of the requests and improve caching, and in the worst case, it should have about the same caching, bandwidth, and performance cost.

[Geometrically]

Merged spritesheets,

I think for versions/snapshots/archaic versions should be initialized into the page on load, and like you said snapshots and archaic will only be added when the user clicks on it, We can have a simple API endpoint which gets these versions for us instead of a costly request to launcher meta. I can get into implementing this once the base api is decided on.

[Geometrically]

For 1.16 filtering, this is because CurseForge hasn't added 1.16 to their API, as their launcher doesn't support it. I don't think it will be added soon due to the merger

[Aeledfyr]

Closing this, as besides for the versions list API, this is no longer related the this repository. This was also specific to the old frontend, so it does not directly translate to the new frontend.