MSUTeam / MSU

Modding Standards and Utilities for Battle Brothers
21 stars 4 forks source link

feat: add persistent storage which doesn't require BBParser #331

Closed Enduriel closed 5 months ago

Enduriel commented 9 months ago

So this is a big one.

Basic interface is of course in the persistent_data_mod_addon, should be self-explanatory.

The backend stuff is a bit more interesting. Basically, I emulate the entire backend system serialization system so that we can have a soft error in case of issues and the game doesn't hard crash if a mod messes something up. The performance of this is frankly still pretty great and not an issue at all in my testing (though I admittedly haven't quantified how long stuff takes)

The way the data is structured is a bit strange but here's the basic concept. Data is serialized by storing the type followed by the data, so when deserializing we can always safely decode it and identify when we have a type we don't know. When deserializing we immediately instantiate the correct serialiation data type for the type we read immediately after reading the type.

More complex data inherits from CustomSerializationDatawhereas basic data inherits from BasicSerializationData. CustomSerializationDatahas the main chunk of the (de)serialization logic, because each SerializationDatais responsible for it's own (de)serialization, and since CustomSerializationDatamust be able to (de)serialize all data, it must contain the logic for that.

At the top level, when createFile is called with some data, it is used to instantiate an ArraySerializationData which then builds its internal array with all the passed data converted into SerializationData and prepared for serialization. The one exception is when (de)serializing BB Objects, those must use a DataArrayData to store and read their data. Imo this is best explained with a simple example:

local myData = {
    string = "someString",
    int = 124,
    float = "1.1241241",
    player = ::MSU.Class.DataArrayData(),
    table = {
        nestedInt = 1
    }
}

local myPlayer //create or grab player
myPlayer.onSerialize(myData.player.getSerializationEmulator()) // fill the DataArrayData with player data

mod.PersistentData.createFile("someName", myData) // the created file will be MSU#mod_id#someName
// it will exist in game saves but will not be shown in load/save screens due to the hook on UIDataHelper
// the data will first be fully converted using the StrictSerializationEmulator
// to ensure there will be no hard errors during file creation
// and then that will write it

// now to get data back we simply do
local readData = mod.PersistenData.readFile("someName")
// this should now equal to myData
// to get player back we can do
local readPlayer // create player from temp roster or smtng
readPlayer.onDeserialize(readData.player.getDeserializationEmulator())

The point is that except for DataArrayData, all the data can be infinitely nested safely without issues, which allows for completely arbitrary data organization.

The StrictSerDeEmulators also have some pretty cool uses for debugging stuff like the legends save corruption because they allow you to basically serialize the entire game into a single DataArrayData safely, guarantee either a useful error or a safe save to an external file which could then be read in a different instance which imo has huge implications.

Necro has also obviously been testing this in his origin maker which (along with MSU's mod settings) are the two major mods which would immediately hugely benefit from this system.

Enduriel commented 5 months ago

Superseded by #341