Sync stats with save files

[elyashivhazan]

Hi i played a lot of games with goldberg but didnt know it can support stats.

One of the games is Celest it stores how much I died and used dashes but because goldbers didnt recorded the stats is it passable to use this fork to sync the stats from a save? is it depended on a game or something else?

Thanks for your help.

[otavepto]

Stats has always been saved in your global saves directory (unless overridden by the local/portable saves option)

Global saves location: https://github.com/otavepto/gbe_fork/blob/dev/post_build/README.release.md#savessettings-location

Local/Portable saves: https://github.com/otavepto/gbe_fork/blob/dev/post_build/README.release.md#portable-saves

Here's an example

If the game calls Steam_User_Stats::SetStat() then the emu will generate the files shown above and load them on next runs, otherwise it won't :/ https://github.com/otavepto/gbe_fork/blob/987d29084c168929a693514e04fd4b6a00e6f72b/dll/steam_user_stats.cpp#L873-L876

Use the debug build, play the game for a while and do something that increases/changes the stat, and post the debug log file STEAM_LOG.txt

[otavepto]

You must also generate stats.txt for this functionality to work, use the tool generate_emu_config and login with your steam account. Without this file the entire stats functions will refuse to save anything. It will look like this:

STAT_LUGLOX_KILLS=int=0
STAT_GRUNT_EYEPOP=int=0
STAT_HUNT_CHALS=int=0
STAT_NWARP_BASES=int=0
STAT_TRADING_CARDS=int=0
STAT_FIND_ALL_DLC1_CARDS=int=0
STAT_FIND_ALL_DLC1_PACKAGES=int=0
STAT_CHARGED_BALL_KILLS=int=0
STAT_HARPER_GLOB_KILLS=int=0

The syntax is simple: https://github.com/otavepto/gbe_fork/blob/dev/post_build/README.release.md#stats But it's really difficult/boring to do it manually

[elyashivhazan]

i meant from game saves, i played without the stats.txt for a while and now i wonder if its passable to like sync my number of deaths etc

[elyashivhazan]

@otavepto hi im sorry for pinging but i couldnt find this info on the internet.

i have this data from my steam account using the web api:

{
  "STAT_Kills": 701,
  "STAT_LevelsCompleted_1to32": 4294966270,
  "STAT_LevelsCompleted_33to64": 4294967295,
  "STAT_LevelsCompleted_65to96": 4294967295,
  "STAT_LevelsCompleted_97to128": 536870527,
  "STAT_LevelsWon_1to32": 4294966270,
  "STAT_LevelsWon_33to64": 4294967295,
  "STAT_LevelsWon_65to96": 4294967295,
  "STAT_LevelsWon_97to128": 536870527,
  "STAT_SniperKills": 1
}

could u tell me what binary format the emu uses for stats?

ik python but not c++ if i would, i would search it my self

edit: as far as i can tell this is stored as little-endian int but that only for int stats and its from celecst stats files that i have

[otavepto]

Hello, yes sure. The format is just raw binary in little endian format Example, here the stat name is busted and the actual value is 2 (int32)

1. Change the stat name to lowercase, let's say my_stat
2. Create a new raw/empty file %appdata%/GSE Saves/<APPID>/stats/my_stat
3. Define the datatype of that stat, either int32 or float, this is important because Valve APIs make a distinction between them.
4. Write your data inside that file in little endian format

Coincidentally with PR #236 you'll have a new option allow_unknown_stats which you can use with the debug build to identify the datatype of the stat when the game requests it. Or just let the tool generate_emu_config generate the default file stats.txt which would contain the datatype as I mentioned above, and encode that accordingly.

These high values 4294966270 are worrying, the APIs are either int32 or float, float can handle it, but your stats look like uint32, even then you're getting too close to the max value.

[elyashivhazan]

so i have this python function

def stats():
    file = f"./ach/stat_{appid}.json"
    stat_data = f"./stats/stats_{appid}.txt"

    if not os.path.isfile(stat_data):
        print("No stats")
        exit()

    with open(file, "r") as f, open(stat_data) as stat_data:
        data = json.load(f)
        stat_data = stat_data.readlines()

        try:
            d = data["playerstats"]["stats"]
        except KeyError as e:
            d = None
            print("No stats")
            exit()

    dir = ".stats"
    if not os.path.isdir(dir):
        os.mkdir(dir)

    for i in d:
        name = i["name"]
        file = name.lower()

        lines = []
        for line in stat_data:
            if line.lower().startswith(f"{file}="):
                lines.append(line.strip())
        if len(lines)>1:
            print("found more then one")
            exit()

        stat_type=lines[0].split("=")[1]

        value = i["value"]
        if stat_type == "int":
            b = struct.pack("<I", value)
        elif stat_type == "float":
            print(f"WIP:float: {file}")
        elif stat_type == "avgrate":
            print(f"WIP:avgrate: {file}")
        else:
            print(f"Unknown type: {stat_type}")

        with open(os.path.join(dir, file), "wb") as f:
            f.write(b)

i manage to convert to int (that was very easy) but i just couldnt for the life of me understand how to convert the float number to binary. i dont have a float stat file nor avgrate from past saves so dose it need to be also 32bit value? because what i tried to do is to convert float to 4 bytes in pyhon

i dont know if im correct but avgrate should always be float right? (looking at the emu config)

> grep avgrate *
stats_207140.txt:speed_avg=avgrate=0.0
stats_24240.txt:m4_accuracy=avgrate=0.0
stats_24240.txt:mp5_accuracy=avgrate=0.0
...

[otavepto]

According to C++ ref: https://en.cppreference.com/w/cpp/language/types

float — single precision floating-point type. Usually IEEE-754 binary32 format

Which leads to this: https://stackoverflow.com/questions/6286033/reading-32-bit-signed-ieee-754-floating-points-from-a-binary-file-with-python

Here's a C++ program to simulate writing a float value (raw binary), then displaying the file content

int main(int argc, char* argv[])
{

    // assume this is the file we'll write to
    char my_stat_file[4] = { 0 };

    // whatever value you want
    float some_value = 3.14f;

    // write the value to the file
    memcpy(my_stat_file, &some_value, sizeof(some_value));

    // display the file content
    for (char byte : my_stat_file) {
        std::cout << "0x" << std::hex << std::uppercase << (byte & 0xFF) << ", ";
    }

    return 0;
}

Hope that helps in anyway :/

[elyashivhazan]

so apparently i had it all the time but python printed it with ascii char instead of \x--, thanks a lot for your help. btw c++ dosnt look that bad at least from that small code it seams to be like pipes in unix just backwards

again thanks a lot