Can I read a savefile from Python?

[ikorotkin]

Hello,

First of all, many thanks for the work you have done. It's a great project, thank you for supporting it!

I am not familiar with TypeScript, but out of curiosity, I have been trying to play around with the savefiles (sandbox*) using Python. I managed to decompress a test savefile into a proper JSON object:

{
    "m_InternalName": "sandbox11",
    "m_DisplayName": "TestH",
    "m_Timestamp": "2023-02-01 23:33:42",
    "m_GameMode": "SANDBOX",
    "m_GameId": 11,
    "m_InstalledOptionalContent": [
        "DLC01"
    ],
    "m_Version": 100,
    "m_Changelist": 106075,
    "m_Dict": {
        "boot": "H3sibV9TY2VuZU5hbWUiOiJRdW9uc2V0R2FzU3RhdGlvAm4ifQ==",
        "global": "H3sibV9WZXJzaW9uIjo0LCJtX0dhbWVNYW5hZ2VyU2VyH2lhbGl6ZWQi ... =",
        ...
    }
}

It looks good except for the m_Dict field, which contains quite readable keys (such as global) but unreadable values (all values start from H3sibV9 for some reason and end with =). Can I ask you a question, what are these values and how can I de-code them into something meaningful?

[FINDarkside]

Sure, here's the steps to decode the m_Dict items:

• Convert it from base64 to binary data
• Decompress the binary data with the same algorithm you decompressed the save file
• Parse the resulting JSON string

Let me know if you have more questions! :blush:

[ikorotkin]

Thank you so much for your super quick response!

I just followed your advice, and it worked! It's very interesting, I didn't expect it was compressed twice :)

[FINDarkside]

It's very interesting, I didn't expect it was compressed twice :)

Yeah the format isn't that smart to be honest. There's also multiple layers of JSON serialization which results in wasted space because JSON.stringify(JSON.stringify(JSON.stringify({foo: 'bar'}))) is '"\\"{\\\\\\"foo\\\\\\":\\\\\\"bar\\\\\\"}\\""' instead of just {"foo":"bar"}. I once tested it out and managed to reduce the save file size to just 19% of the original by just getting rid of the nested serialization. Actually the raw non-nested and non compressed JSON file was smaller than this "compressed" save :sweat_smile:

[ikorotkin]

Yeah, I noticed a ridiculous amount of \ in the output too. Not sure how they end up with that. It must be a pain to write this file back after making changes.

The fact that a proper non-nested and non-compressed JSON was smaller than this double-compressed save is funny indeed :) The devs had a perfect chance to improve the saves when they released the DLC recently. Old saves became incompatible with the new game anyway.

Well, but at least the saves are readable, thanks to you and your project! 👍

[FINDarkside]

The devs had a perfect chance to improve the saves when they released the DLC recently. Old saves became incompatible with the new game anyway.

Yeah the changes to the same format were very minor in that update. The did do one improvement though, they finally started using Base64 for the binary data. Earlier they user number arrays. So for example "Old saves became incompatible with the new" in base64: T2xkIHNhdmVzIGJlY2FtZSBpbmNvbXBhdGlibGUgd2l0aCB0aGUgbmV3 was saved as [84,50,120,107,73,72,78,104,100,109,86,122,73,71,74,108,89,50,70,116,90,83,66,112,98,109,78,118,98,88,66,104,100,71,108,105,98,71,85,103,100,50,108,48,97,67,66,48,97,71,85,103,98,109,86,51] in the JSON. That's +175% to the file size right there :smile:

[FINDarkside]

I'll close this issue to cleanup open issues