Reloaded-Project / Reloaded-III

[WIP] Formal Specification for the Next Iteration of Reloaded
https://reloaded-project.github.io/Reloaded-III/
8 stars 1 forks source link

TEMP/WIP [IDEA] 'Offline Mods' & Collections & Events #6

Closed Sewer56 closed 6 months ago

Sewer56 commented 1 year ago

[Just a placeholder of ideas I have while working on other stuff (hooks library), written down so I don't forget; ] [This text is very rough, very far from final]

Summary

Allow the user to download only mod metadata; i.e. 'download just enough for user to be able to configure and enable their mod(s)'.

Reminder: Reloaded must remain website/service agnostic.

When is a mod 'offline'

A mod is considered offline when the files defined in Package Manifest are missing.

Where Does user Get Package Manifests?

The main source of package manifests is from the user's own 'collection'. Every single user profile creates an implicit 'collection'.

A 'collection' consists of the following:

The expected size of a large-ish collection (~300 mods) is around 1MiB (after SOLID compression).

Each collection is a typical reloaded package, that means, it can be packed and uploaded at any time. User collection packages will have random characters appended to the end to avoid potential duplicates.

The contents of this collection are updated when the user launches a game or closes the application, and are stored and compressed inside a single serializable json (or equivalent) file. This improves I/O performance, and file size.

After application exit, user collections should be synced to user's backup/restore service, such as Cloud Storage, Steam Cloud, or even uploaded as a mod, etc.

Image Availability

Collections ship without images. In order to see intended images for mods in the collection (including thumbnails); the user will have to download the mods.

Under the launch button, a 'download' button should be shown in the UI, which will start download of all enabled mods. There will eventually be an optional 'auto-download' option.

In order to make image downloading less painful, all images will be compressed with JPEG-XL, which should reduce thumbnails down to 75-150KB and 1080p screenshots to <400KB.

Image Resolve (on Supported Sites)

On supported websites, R3 should partial download the mods (download just the images from entire mod archives).

This should be possible if one of the following archive formats is used:

Ideally we would use NX archives (they're perfect for this), however, not many sites would allow us to upload these (due to their virus scanning requirements; unsupported archive format). In any case, if a partial download is possible, we will start fetching images from the mod archives in the background directly.

Image Resolve (Fallback for Supported Sites)

On supported websites, R3 will fetch the images for non-downloaded mods from the mod's uploaded images on the supported modding site.

The image with the closest aspect ratio on a given modding site should be downloaded as the temporary thumbnail for a mod.

Temporary thumbnails should have an indicator, preferably in a corner of the image; when the user hovers over it, they should be informed that the image is a placeholder.

Image Resolver (via Service)

Eventually the Reloaded3 version of the Reloaded Index could download any new mods and cache the images present into GitHub repository or VPS.

This will however only be performed if the situation turns bad enough that the need for this arises.

Use Cases

Collection Technical Details

Collections are encoded as a set of 'events'.

Header should look something like:

u32: NumEvents
[.. Events ..]

Each event has consists of:

u8: EventType
[.. Event Data, Padded to Next 8 Bytes ..]

To optimize for read speeds, all events are padded to 8 bytes (to align with native 64-bit word size). The user can roll back to any previous loader state by 'replaying' the events from zero.

Example events include:

Each collection is stored inside a folder on FileSystem. Mods directly load/save configs to FileSystem (for backcompat reasons, each mod should have its own folder).

When a collection is shared as a package, a snapshot is created at the current point, and history is erased, to keep download package small. This means doing a cleanup of dead data.

Technical Challenges

non-GitHub

non-NexusMods (GameBanana, GitHub, etc.)

NexusMods

Sewer56 commented 6 months ago

This was actually done fairly recently.