fixed "unknown" byte placement in "wall" action

[Skazu]

I rearranged the unknown bytes in the "wall" action, it looks like those are individual zero bytes right before each of start{x,y} and end{x,y}. So in the current implementation start_x and end_x are always 0, while the "unk" field holds the actual values:

import os
from mgz import header, body

with open("var/savegames/MP Replay v101.101.43210.0 @2020.12.21 141238 (3).aoe2record", "rb") as data:
    eof = os.fstat(data.fileno()).st_size
    header.parse_stream(data)
    body.meta.parse_stream(data)
    while data.tell() < eof:
        op = body.operation.parse_stream(data)

        if op.type == "action" and op.action.type == "wall":
            print(op)
            break

Result:

Container: 
    type = action (total 6)
    start = 1110834
    op = 1
    length = 28
    action = Container: 
        type_int = 105
        type = wall (total 4)
        selected = 1
        player_id = 3
        start_x = 0
        start_y = 30
        end_x = 0
        end_y = 105
        unk = $\x00c\x00 (total 4)
        building_id = 72
        flags = \x01\x01\x00\x00 (total 4)
        unit_ids = ListContainer: 
            33463
    end = 1110874

With the fix the result looks like this:

Container: 
    type = action (total 6)
    start = 1110834
    op = 1
    length = 28
    action = Container: 
        type_int = 105
        type = wall (total 4)
        selected = 1
        player_id = 3
        start_x = 30
        start_y = 105
        end_x = 36
        end_y = 99
        building_id = 72
        flags = \x01\x01\x00\x00 (total 4)
        unit_ids = ListContainer: 
            33463
    end = 1110874

I'm unsure if the unknown bytes are always \x00, this was the case in all my tests, but to prevent any errors i choose to use Bytes(1) instead of Const('\x00').

[happyleavesaoc]

I believe this is only correct for DE. It breaks AOC wall action parsing. I'll look for a solution if you're not up for ensuring AOC compat - I know that's not your focus.

[Skazu]

I tried to be as careful as possible and added the bytes just under the same circumstances as before, but moved the positions.

I've testet all records in tests/recs (*) as well as a lot new DE records and some recorded games from aocrecs.com.

Can you maybe provide the record which triggers the error? Then i will have a look at it.

(*) Result of tests/recs files:

-- PARSING: tests/recs/de-13.07.aoe2record
Version.DE
Container: 
    type = action (total 6)
    start = 544580
    op = 1
    length = 28
    action = Container: 
        type_int = 105
        type = wall (total 4)
        selected = 1
        player_id = 1
        start_x = 73
        start_y = 86
        end_x = 73
        end_y = 86
        building_id = 72
        flags = \x00\x01\x00\x00 (total 4)
        unit_ids = ListContainer: 
            2765
    end = 544620

-- PARSING: tests/recs/size-255.mgz
Version.USERPATCH15
Container: 
    type = action (total 6)
    start = 415878
    op = 1
    length = 20
    action = Container: 
        type_int = 105
        type = wall (total 4)
        selected = 1
        player_id = 1
        start_x = 55
        start_y = 42
        end_x = 55
        end_y = 45
        building_id = 72
        unit_ids = ListContainer: 
            1929
    end = 415910

-- PARSING: tests/recs/de-13.03.aoe2record
Version.DE
Container: 
    type = action (total 6)
    start = 396544
    op = 1
    length = 28
    action = Container: 
        type_int = 105
        type = wall (total 4)
        selected = 1
        player_id = 2
        start_x = 82
        start_y = 71
        end_x = 82
        end_y = 71
        building_id = 72
        flags = \x00\x01\x00\x00 (total 4)
        unit_ids = ListContainer: 
            1003
    end = 396584

-- PARSING: tests/recs/de-13.13.aoe2record
Version.DE
Container: 
    type = action (total 6)
    start = 1129335
    op = 1
    length = 28
    action = Container: 
        type_int = 105
        type = wall (total 4)
        selected = 1
        player_id = 2
        start_x = 100
        start_y = 66
        end_x = 100
        end_y = 66
        building_id = 72
        flags = \x00\x01\x00\x00 (total 4)
        unit_ids = ListContainer: 
            1580
    end = 1129375

-- PARSING: tests/recs/de-13.17.aoe2record
Version.DE
Container: 
    type = action (total 6)
    start = 785742
    op = 1
    length = 28
    action = Container: 
        type_int = 105
        type = wall (total 4)
        selected = 1
        player_id = 3
        start_x = 133
        start_y = 54
        end_x = 133
        end_y = 54
        building_id = 72
        flags = \x01\x01\x00\x00 (total 4)
        unit_ids = ListContainer: 
            10973
    end = 785782

-- PARSING: tests/recs/de-13.20.aoe2record
Version.DE
Container: 
    type = action (total 6)
    start = 531554
    op = 1
    length = 28
    action = Container: 
        type_int = 105
        type = wall (total 4)
        selected = 1
        player_id = 1
        start_x = 21
        start_y = 21
        end_x = 21
        end_y = 21
        building_id = 72
        flags = \x00\x01\x00\x00 (total 4)
        unit_ids = ListContainer: 
            1592
    end = 531594

-- PARSING: tests/recs/small.mgz
Version.USERPATCH15
Container: 
    type = action (total 6)
    start = 521189
    op = 1
    length = 20
    action = Container: 
        type_int = 105
        type = wall (total 4)
        selected = 1
        player_id = 1
        start_x = 59
        start_y = 26
        end_x = 53
        end_y = 26
        building_id = 72
        unit_ids = ListContainer: 
            2104
    end = 521221

-- PARSING: tests/recs/aoc-1.0.mgx
Version.AOC10
Container: 
    type = action (total 6)
    start = 372950
    op = 1
    length = 32
    action = Container: 
        type_int = 105
        type = wall (total 4)
        selected = 1
        player_id = 2
        start_x = 20
        start_y = 15
        end_x = 20
        end_y = 15
        building_id = 72
        unit_ids = ListContainer: 
            1696
    end = 372994

-- PARSING: tests/recs/aoc-1.0c.mgx
Version.AOC10C
Container: 
    type = action (total 6)
    start = 402275
    op = 1
    length = 20
    action = Container: 
        type_int = 105
        type = wall (total 4)
        selected = 1
        player_id = 1
        start_x = 73
        start_y = 14
        end_x = 73
        end_y = 18
        building_id = 72
        unit_ids = ListContainer: 
            2108
    end = 402307

-- PARSING: tests/recs/de-13.08.aoe2record
Version.DE
Container: 
    type = action (total 6)
    start = 649272
    op = 1
    length = 28
    action = Container: 
        type_int = 105
        type = wall (total 4)
        selected = 1
        player_id = 1
        start_x = 82
        start_y = 61
        end_x = 82
        end_y = 61
        building_id = 72
        flags = \x00\x01\x00\x00 (total 4)
        unit_ids = ListContainer: 
            2851
    end = 649312

-- PARSING: tests/recs/de-12.97-8byte-tile.aoe2record
Version.DE
Container: 
    type = action (total 6)
    start = 327413
    op = 1
    length = 28
    action = Container: 
        type_int = 105
        type = wall (total 4)
        selected = 1
        player_id = 2
        start_x = 24
        start_y = 22
        end_x = 24
        end_y = 22
        building_id = 72
        flags = \x00\x01\x00\x00 (total 4)
        unit_ids = ListContainer: 
            7349
    end = 327453

-- PARSING: tests/recs/de-12.97-6byte-tile.aoe2record
Version.DE
Container: 
    type = action (total 6)
    start = 401298
    op = 1
    length = 28
    action = Container: 
        type_int = 105
        type = wall (total 4)
        selected = 1
        player_id = 2
        start_x = 77
        start_y = 79
        end_x = 71
        end_y = 79
        building_id = 72
        flags = \x00\x00\x00\x00 (total 4)
        unit_ids = ListContainer: 
            1915
    end = 401338

-- PARSING: tests/recs/test.mgz
Version.USERPATCH15
Container: 
    type = action (total 6)
    start = 494477
    op = 1
    length = 20
    action = Container: 
        type_int = 105
        type = wall (total 4)
        selected = 1
        player_id = 2
        start_x = 62
        start_y = 41
        end_x = 62
        end_y = 41
        building_id = 72
        unit_ids = ListContainer: 
            2156
    end = 494509

-- PARSING: tests/recs/de-13.06.aoe2record
Version.DE
Container: 
    type = action (total 6)
    start = 9626587
    op = 1
    length = 60
    action = Container: 
        type_int = 105
        type = wall (total 4)
        selected = 9
        player_id = 1
        start_x = 84
        start_y = 55
        end_x = 81
        end_y = 56
        building_id = 117
        flags = \x01\x01\x00\x00 (total 4)
        unit_ids = ListContainer: 
            7834
            7287
            7292
            7567
            7517
            7875
            7763
            7816
            7057
    end = 9626659

-- PARSING: tests/recs/de-13.15.aoe2record
Version.DE
Container: 
    type = action (total 6)
    start = 393446
    op = 1
    length = 28
    action = Container: 
        type_int = 105
        type = wall (total 4)
        selected = 1
        player_id = 1
        start_x = 87
        start_y = 22
        end_x = 87
        end_y = 22
        building_id = 72
        flags = \x00\x01\x00\x00 (total 4)
        unit_ids = ListContainer: 
            3342
    end = 393486

-- PARSING: tests/recs/aok-2.0a.mgl
Version.AOK
Container: 
    type = action (total 6)
    start = 383980
    op = 1
    length = 20
    action = Container: 
        type_int = 105
        type = wall (total 4)
        selected = 1
        player_id = 2
        start_x = 56
        start_y = 76
        end_x = 48
        end_y = 76
        building_id = 72
        unit_ids = ListContainer: 
            1915
    end = 384012

-- PARSING: tests/recs/de-13.34.aoe2record
Version.DE
Container: 
    type = action (total 6)
    start = 592984
    op = 1
    length = 28
    action = Container: 
        type_int = 105
        type = wall (total 4)
        selected = 1
        player_id = 1
        start_x = 45
        start_y = 40
        end_x = 50
        end_y = 31
        building_id = 72
        flags = \x00\x01\x00\x00 (total 4)
        unit_ids = ListContainer: 
            8869
    end = 593024

-- PARSING: tests/recs/up-1.4.mgz
Version.USERPATCH14
Container: 
    type = action (total 6)
    start = 389892
    op = 1
    length = 20
    action = Container: 
        type_int = 105
        type = wall (total 4)
        selected = 1
        player_id = 1
        start_x = 51
        start_y = 80
        end_x = 38
        end_y = 93
        building_id = 72
        unit_ids = ListContainer: 
            1892
    end = 389924

No errors, everything looks plausible.

[happyleavesaoc]

I confirmed that you are correct. I spot-checked earlier today and probably misread a number. Apologies!

Anyway, I think the DE version of this action has unsigned shorts (Int16ul in construct) instead of single bytes to account for maps with dimensions > 255 like the "ludikris" size. That means those "extra" bytes will not always be 0. I recommend moving the coordinates inside the If and doing the DE style with Int16ul and the legacy style with Byte.

But, it's little-endian so the "normally zero" byte would have to be after the existing values. The Padding(1) byte after the final y is probably a 0 and the real padding is before the first x.

[Skazu]

Yes! Thats it! Thanks for pointing that out. I've changed the code accordingly.

Output for walling on ludicrous-sized maps at positions > 255:

Container: 
    type = action (total 6)
    start = 3394952
    op = 1
    length = 32
    action = Container: 
        type_int = 105
        type = wall (total 4)
        selected = 2
        player_id = 2
        start_x = 236
        start_y = 420
        end_x = 240
        end_y = 435
        building_id = 117
        flags = \x00\x01\x00\x00 (total 4)
        unit_ids = ListContainer: 
            182234
            182315
    end = 3394996

looks plausible.

All the test records are still parsing without errors and the coordinates are looking like before.

[happyleavesaoc]

Thanks, looks great now!