Research: Network streaming of game (assets)

[voidderef]

In GitLab by @icex2 on May 4, 2020, 22:05

Context Initial idea proposed by a very special shark, instead of keeping all data available on a local persistent disk, keep it stored on a remote server. When the game wants to read a file, a backend takes care of fetching files from the remote server, caching and evicting files from cache.

Why? Usually the number of files the game needs at a given moment is a rather small subset compared to the total size of the game asset folder. This feature would eliminate the requirement to download any game data prior running the game. Multiple strategies for data retrival are possible here:

• On-demand only with a small cache -> Potentially even keep everything in-memory only. Settings could be virtualized and stored on a server as well.
• Determine which files are used often and keep them cached on a local persistent disk. Reduces network traffic and loading times.
• Full download in the background: Download everything to launch the game on-demand including any songs. Simultaneously, run download tasks in the background to retrieve all game data -> impact on game's performance when downloading files during main gameplay?

Actions I am quite sure this is not the first time such an idea pops up in the context of game/application asset distribution. Therefore, ask good old Google if there is anything out there that fits the above requirements. Figure out advantages and drawbacks and write them down in this task.

If an existing solution can be utilized, create a proof of concept prototype.

If no existing solution exists or fits well enough, consider a custom solution. Considering this is on a file/file system level, I don't think this needs to be integrated into pumptools and can actually work as it's own standalone application/project.

Consider a design that might also work with other applications/games with a similar environment, e.g. bemani games.

Remark: Ultimately, the tool would bootstrap the entire game in a launcher style fashion, i.e. even download the game executable, libraries and pumptools using a configuration file describing what the launcher is supposed to run.

How to implement this? Well, that's the thing to do: Come up with potential options including their benefits and drawbacks.

/minddump

[voidderef]

In GitLab by @icex2 on May 4, 2020, 22:06

changed the description

[voidderef]

In GitLab by @icex2 on May 4, 2020, 22:16

changed the description

[voidderef]

In GitLab by @icex2 on May 4, 2020, 22:41

Some ideas for potential solutions proposed by people in a chat:

rsync files prior game launch

Pro:

• Full pre-load and automatic update check before launch
• Uses existing tooling, rather simple to implement

Con:

• No on-demand loading during game runtime
• No caching/background loading during game runtime

POC AVS remotefs for bemanigames

Pro:

• Full on-demand loading of assets from remote server
• Initial tests were promising regarding performance (on a good network?)

Con:

• Based on AVS for bemanigames, limited re-usability