Monstrofil / replays_unpack

51 stars 19 forks source link

Investigate unknown packet types #18

Open Monstrofil opened 2 years ago

Monstrofil commented 2 years ago

E.g. for wows:

TIME: 0.0 TYPE: 0x16 SIZE: 12 DATA: <_io.BytesIO object at 0x000001ADFF4BABE8> 22
TIME: 0.0 TYPE: 0x10 SIZE: 1 DATA: <_io.BytesIO object at 0x000001AD809DCB88> 16
TIME: 0.0 TYPE: 0xf SIZE: 8 DATA: <_io.BytesIO object at 0x000001AD809DCE28> 15
TIME: 0.0 TYPE: 0x26 SIZE: 4 DATA: <_io.BytesIO object at 0x000001AD809FA108> 38
TIME: 0.0 TYPE: 0x25 SIZE: 10 DATA: <_io.BytesIO object at 0x000001AD809DCD68> 37
TIME: 0.0 TYPE: 0x10 SIZE: 1 DATA: <_io.BytesIO object at 0x000001AD809DCF48> 16
TIME: 0.0 TYPE: 0xe SIZE: 8 DATA: <_io.BytesIO object at 0x000001AD809FADC8> 14
TIME: 0.0 TYPE: 0x31 SIZE: 8 DATA: <_io.BytesIO object at 0x000001AD809FABE8> 49
TIME: 0.0 TYPE: 0x13 SIZE: 0 DATA: <_io.BytesIO object at 0x000001AD8095A5E8> 19
TIME: 0.3901594877243042 TYPE: 0x24 SIZE: 56 DATA: <_io.BytesIO object at 0x000001AD80A265E8> 36

TIME: 0.0 TYPE: 0xffffffff SIZE: 16 DATA: <_io.BytesIO object at 0x000001AD80A4A6A8> 4294967295

The last one is definitely EndGame or something like that,

razaqq commented 2 years ago

Here are some infos i found out about unknown packet types:

These two occur hundreds of times, but their payload never changes:

Some more:

dacite commented 2 years ago

I'm not sure if this is already known, but 0x16 is extremely useful for World of Tanks because it can tell us when the battle actually starts. This is because the time inside the replay (top right corner) does not reflect the timestamp of a packet when it occurs. With this packet we can calculate the actual replay time of the packet and this can help with decoding packets if we watch the replay along with it.

0x16 occurs exactly 3 times for all .wotreplays I tested (2016 - 2022). It's payload is 4 bytes always and seems to be a uint32_t. From my testing it looks like the third time it occurs is when the battle actually starts (countdown ends). We can also tell when the battle starts by just looking at the payload:

I derived this from the following code from WoT (res/scripts/common/constants.py):

class ARENA_PERIOD:
    IDLE = 0
    WAITING = 1
    PREBATTLE = 2
    BATTLE = 3
    AFTERBATTLE = 4

It's possible that this might not be the exactly correct data structure but if the payload is 0x03, we can be pretty certain that packet's timestamp is when the battle starts.

Here is an example output showing all occurrences of 0x16 for 4 replays:

"20220306_0941_france-F108_Panhard_EBR_105_45_north_america.wotreplay"
[04000000 16000000 00000000 01000000] Timestamp: 00000000 = 0.0 seconds
[04000000 16000000 85EB8940 02000000] Timestamp: 85EB8940 = 4.3099999 seconds
[04000000 16000000 713D0942 03000000] Timestamp: 713D0942 = 34.310001 seconds

"20200502_0355_sweden-S26_Lansen_C_06_ensk.wotreplay"
[04000000 16000000 00000000 01000000] Timestamp: 00000000 = 0.0 seconds
[04000000 16000000 52B83641 02000000] Timestamp: 52B83641 = 11.42 seconds
[04000000 16000000 CD4C2542 03000000] Timestamp: CD4C2542 = 41.325001 seconds

This one is bit weird because the timestamp doesn't line up with others:
"20160603_2346_germany-G120_M41_90_GrandFinal_11_murovanka.wotreplay"
[04000000 16000000 00000000 01000000] Timestamp: 00000000 = 0.0 seconds
[04000000 16000000 00000000 02000000] Timestamp: 00000000 = 0.0 seconds
[04000000 16000000 3333C741 03000000] Timestamp: 3333C741 = 24.9 seconds

This is one is around the same time as the above but the timestamp make sense:
"20160624_2021_usa-A35_Pershing_11_murovanka.wotreplay"
[04000000 16000000 00000000 01000000] Timestamp: 00000000 = 0.0 seconds
[04000000 16000000 33331340 02000000] Timestamp: 33331340 = 2.3 seconds
[04000000 16000000 33330142 03000000] Timestamp: 33330142 = 32.299999 seconds

Therefore, we can calculate the replay time of a packet as following:

battle_start_time = timestamp of packet 0x16 with payload of 0x03 // we can either look at the payload or the 3rd 0x16 occurrence

packet_replay_time = packet_timestamp - battle_start_time

Convert packet_replay_time to MM:SS timer 

Algorithm I use in Rust (to get the battle start time):

fn get_battle_start_time(packet_stream: PacketStream) -> f32 {
    for packet in packet_stream.into_iter() {
        if packet.get_type() == 0x16 {
            if packet.get_payload().read_u32::<LE>().unwrap() == 3 {
                return packet.get_time();
            }
        }
    }
    return -1.0;
}

A notable exception to this is the Chat packet(I think 0x23 for WoT). Probably because you can send chats before the battle actually starts.

Unfortunately I cannot test how this works in WoWS as I don't play it but a similar thing might exist.

razaqq commented 2 years ago

You might want to have a look here: https://github.com/lkolbly/wows-replays/issues/14 There is a some more info on other packets there

Monstrofil commented 2 years ago

I'm not sure if this is already known, but 0x16 is extremely useful for World of Tanks because it can tell us when the battle actually starts. This is because the time inside the replay (top right corner) does not reflect the timestamp of a packet when it occurs. With this packet we can calculate the actual replay time of the packet and this can help with decoding packets if we watch the replay along with it.

I don't think this packet exist in wows. The timing data in wows is handled by extra entity called BattleLogic, which has information about time and battle state.

The 0x16 seems to be used for version data: b'\x08\x00\x00\x000,11,3,0'

razaqq commented 2 years ago

yeah 0x16 for wows is explained in detail above

dacite commented 1 year ago

0x2D for WoT seems to represent some sort of version. This is not the same as the Game version (which is 0x18). It can be parsed exactly the same way as the game version packet for both WoT and WoWs. In really old WoT Replays, for example patch 0.9.8, this value is present in 0x29 packet.

Here is a table showing the replay version (WoT version when replay was recorded) and the unknown version Replay Version 0x18 Unknown Version 0x2D or 0x29
1.17.1.2 17, 1, 0, 3
1.6.1.0 17, 1, 0, 2
1.9.1.0 17, 1, 0, 2
0.9.8.0 2, 9, 0, 0
0.9.17.0 15, 3, 1, 0

Any idea what the version is representing? Does something similar happen in WoWs?