Parser output - Discussion and Draft

[PBug90]

Since the output is currently very verbose and not a lot of information can be used efficiently, I would like to suggest a well defined object that models all required information in an easy way so that both humans and machines can interpret it easily. This is a work in progress and not yet complete. Feel free to comment.

{
  "players": [
    {
      "name": "",
      "race": "",
      "raceDetected": "",
      "color": "",
      "units": {
        "summary": {
          "hfoo": 5,
          "hbar": 10
        },
        "order": [
          {
            "ms": 1500,
            "id": "hfoo"
          },
          {
            "ms": 2500,
            "id": "hfoo"
          }
        ]
      },
      "teamid": 2,
      "id": 0,
      "buildings": {
        "summary": {
          "hfoo": 5,
          "hbar": 10
        },
        "order": [
          {
            "ms": 1500,
            "id": "hfoo"
          },
          {
            "ms": 2500,
            "id": "hfoo"
          }
        ]
      },
      "upgrades": {
        "summary": {
          "hfoo": 5,
          "hbar": 10
        },
        "order": [
          {
            "ms": 1500,
            "id": "hfoo"
          },
          {
            "ms": 2500,
            "id": "hfoo"
          }
        ]
      },
      "items": {
        "summary": {
          "hfoo": 5,
          "hbar": 10
        },
        "order": [
          {
            "ms": 1500,
            "id": "hfoo"
          },
          {
            "ms": 2500,
            "id": "hfoo"
          }
        ]
      }
    }
  ],
  "gamename": "",
  "matchup": "a well defined, universal matchup string",
  "uuid": "a game uuid that is generated for this particular game",
  "type": "1vs1",
  "creator": "BNet",
  "map": {
    "path": "Maps\\FrozenThrone\\(4)TwistedMeadows.w3x",
    "shortened": "TwistedMeadows",
    "file": "(4)TwistedMeadows.w3x"
  },
  "version": "1.31",
  "settings": {
    "visibility": 0,
    "hideTerrain": true,
    "mapExplored": true,
    "alwaysVisible": true,
    "default": true,
    "teamsTogether": false,
    "randomHero": false,
    "randomRaces": false,
    "referees": false,
    "fixedTeams": true,
    "fullSharedUnitControl": true,
    "speed": 2
  },
  "observers": [
    "observer1",
    "observer2"
  ],
  "chat": [
    {
      "mode": "ALL|ALLY|OBS|REF|PRIVATE",
      "player": "a playername",
      "message": "the chat message"
    }
  ],
  "replayIdentifier": "algorithm tbd"
}

[JFGHT]

In my current website, the parser puts the team inside the player "column" and nowhere else to avoid data duplication (it's a relational database). So, to gather information about the teams you need to analyze every single player in the game but to get the team of a player is quite simple, since it's inside that column.

But, due to the fact that this normalization is something breakable in non-relational databases and, in this is just pure JSON, we could include it in both: each player object and in a separate property at the root of the replay object.

My question now comes with the teams root property: wouldn't it be better to have an object instead of an array for the teams, so we have direct access to team 3, for example? There might be games where teams are not straight forward; Team 2 and Team 3 (lack of team 1). But I'm not 100% sure of how does w3 manages this.

[PBug90]

I dont see any use case of tracking the actual in game team ids. In my opinion the only relevant info is the amount of teams and the actual players that are part of that team. Therefore, I put an individual team as an array inside of the teams array. The individual team then has the actual player array index of the player that is part of that team. That way, you can easily iterate the array by using map or reduce without the need of converting it using Object.keys or Object. The ingame team-ids can be stored together with players though if anyone really needs those.

[JFGHT]

You're losing information that could be useful to someone else in exchange of avoiding 'Object' usage.

Also, saving the actual ingame team-ids inside the player while mantaining the currect teams array would end in data inconsistency; team id stored in player doesn't necessarly match the teams array position.

---

Use case example: checking the allies of a target player.

Your code:

• Search the whole teams array for the target player. Check if there's any other player in that array position.

1. O(n) performance issue.
2. Longer code.

My proposal:

• Get the team id from the player. Go to the property [teamId] in the teams object and check if there are more players than the target.

1. No performance issues.
2. Straight code.

[PBug90]

What I was after in the first place is the decision whether to drop ingame team ids in favor of a predictable order of teams starting with index 0 for the first team, because the ingame team id is non-relevant to external sources. What matters is which player is in which team. You did state that the information was useful to someone else. Can you name a use case where the actual ingame team id is really needed? I can not think of one. Thats why I thought about a compromise and put the actual ingame team id and player id into the player object so the information is there if a third party application really needs to utilize it for some reason.

You are right about the O(n) performance issue as long as the array index of a specific player's team is not known. As soon as it is, it becomes O(1) again. This is the case if you save the team index with the player object, which would be the same as your proposal. You can then still access a players team without performance issues and you would have an abstracted view of teams that is uniform across different replays because it does abstract the ingame team ids.

Another thing I thought about was guaranteeing the sort order of players in the player array to reflect the team memberships which would make it easier to render players in an application view with some kind of for loop or a .map operation, which are very common in the javascript world.

[JFGHT]

Can you name a use case where the actual ingame team id is really needed?

Easy one! Custom games for example. Sheep tag (if I remember correctly) has 2 sticky teams; wolves and sheeps. I guess the developer would like to know about the teams ids to identify wolves and sheeps.

Still, dropping information is never good. You never know what others can do with that data.

[PBug90]

Agree to the point of dropping information isnt good. Even though the parser use case was intended for standard games, I can see your point.

I guess the best option is to be able to plug in a custom function as an optional parameter for the constructor that is being called with the parsed replay data in order to adjust the final output the way you want.

As a consequence, I would prefer ordering players by team in the players array and not include a team property at. Meanwhile, player objects do contain a teamid property that represents the ingame team id and are also guaranteed to be ordered by ingame team ids in ascending order. That way one can construct any team information the way they like with the before mentioned custom finalizing function.

[JFGHT]

This is how my (extremely old) PHP parser is saving the data:

{ ID: 34320, cat: '', Datum: 2018-11-05T20:00:38.000Z, PostName: 'Sheila', SpielModus: '1on1', Slot1Name: 'HaSaN123', Slot1Rasse: 'H', Slot1Details: <Buffer 61 3a 36 3a 7b 73 3a 34 3a 22 74 65 61 6d 22 3b 69 3a 30 3b 73 3a 31 30 3a 22 63 6f 6c 6f 72 5f 68 74 6d 6c 22 3b 73 3a 37 3a 22 23 38 36 45 35 37 33 ... >, Slot1AdvancedDetails: <Buffer 78 da a5 56 4b 6f dc 36 10 fe 2b c2 5e 7a ac f8 92 48 ed c9 06 92 1e 82 16 45 b6 45 8f 06 ad 65 76 09 eb 61 90 92 81 22 d0 7f cf f0 b1 6b ad 4c 3a 48 ... >, Slot2Name: 'Andres.', Slot2Rasse: 'O', Slot2Details: <Buffer 61 3a 36 3a 7b 73 3a 34 3a 22 74 65 61 6d 22 3b 69 3a 31 3b 73 3a 31 30 3a 22 63 6f 6c 6f 72 5f 68 74 6d 6c 22 3b 73 3a 37 3a 22 23 44 39 36 43 42 31 ... >, Slot2AdvancedDetails: <Buffer 78 da a5 57 4b 6f e3 38 0c fe 2b 46 2e 7b 1c 3d 2c 59 76 4e 4d 51 cc 00 83 dd 2d 9a 5d f4 58 a8 89 26 11 e2 d8 85 64 77 b0 18 e4 bf 2f f5 b0 f3 92 d3 ... >, Slot3Name: '', Slot3Rasse: '', Slot3Details: <Buffer 73 3a 30 3a 22 22 3b>, Slot3AdvancedDetails: <Buffer 78 da 2b b6 32 b0 52 52 b2 06 00 07 45 01 97>, Slot4Name: '', Slot4Rasse: '', Slot4Details: <Buffer 73 3a 30 3a 22 22 3b>, Slot4AdvancedDetails: <Buffer 78 da 2b b6 32 b0 52 52 b2 06 00 07 45 01 97>, Slot5Name: '', Slot5Rasse: '', Slot5Details: <Buffer 73 3a 30 3a 22 22 3b>, Slot5AdvancedDetails: <Buffer 78 da 2b b6 32 b0 52 52 b2 06 00 07 45 01 97>, Slot6Name: '', Slot6Rasse: '', Slot6Details: <Buffer 73 3a 30 3a 22 22 3b>, Slot6AdvancedDetails: <Buffer 78 da 2b b6 32 b0 52 52 b2 06 00 07 45 01 97>, Slot7Name: '', Slot7Rasse: '', Slot7Details: <Buffer 73 3a 30 3a 22 22 3b>, Slot7AdvancedDetails: <Buffer 78 da 2b b6 32 b0 52 52 b2 06 00 07 45 01 97>, Slot8Name: '', Slot8Rasse: '', Slot8Details: <Buffer 73 3a 30 3a 22 22 3b>, Slot8AdvancedDetails: <Buffer 78 da 2b b6 32 b0 52 52 b2 06 00 07 45 01 97>, Slot9Name: '', Slot9Rasse: '', Slot9Details: <Buffer 73 3a 30 3a 22 22 3b>, Slot9AdvancedDetails: <Buffer 78 da 2b b6 32 b0 52 52 b2 06 00 07 45 01 97>, Slot10Name: '', Slot10Rasse: '', Slot10Details: <Buffer 73 3a 30 3a 22 22 3b>, Slot10AdvancedDetails: <Buffer 78 da 2b b6 32 b0 52 52 b2 06 00 07 45 01 97>, Slot11Name: '', Slot11Rasse: '', Slot11Details: <Buffer 73 3a 30 3a 22 22 3b>, Slot11AdvancedDetails: <Buffer 78 da 2b b6 32 b0 52 52 b2 06 00 07 45 01 97>, Slot12Name: '', Slot12Rasse: '', Slot12Details: <Buffer 73 3a 30 3a 22 22 3b>, Slot12AdvancedDetails: <Buffer 78 da 2b b6 32 b0 52 52 b2 06 00 07 45 01 97>, Observers: <Buffer 4e 3b>, winner_team: '1', w3type: 'WAR3', version: 'v1.30', length: 1003, gametype: 'unknow', Gateway: 'Northrend', MapName: 'download(4)rLostTemple2.w3m', Beschreibung: '', Filesize: 111, lastIP: '172.69.55.176', Downloads: 3, Verstecken: 0, Kommentaranzahl: 0, ChatLog: <Buffer 78 da bd 99 6f 4f dc 38 10 c6 bf 8a 95 37 bc 3a 14 db f9 8f 74 a7 0a 90 8a 8a 68 55 78 73 3a 5d ef bc 1b ef c6 25 b1 57 b1 03 ac aa 7e f7 1b 67 17 ee ... >, rating: 0, votes: 0, for_rotw_and_top: 0, rotw: 0, top: 0, MapHost: 'WarHost', HashValue: '9194e47bba6fd0bbb72e9dfd50e08649', Tournament: '', Description: '', highlighted: 0, Slot13Name: '', Slot13Rasse: '', Slot13Details: <Buffer >, Slot13AdvancedDetails: <Buffer >, Slot14Name: '', Slot14Rasse: '', Slot14Details: <Buffer >, Slot14AdvancedDetails: <Buffer >, Slot15Name: '', Slot15Rasse: '', Slot15Details: <Buffer >, Slot15AdvancedDetails: <Buffer >, Slot16Name: '', Slot16Rasse: '', Slot16Details: <Buffer >, Slot16AdvancedDetails: <Buffer >, Slot17Name: '', Slot17Rasse: '', Slot17Details: <Buffer >, Slot17AdvancedDetails: <Buffer >, Slot18Name: '', Slot18Rasse: '', Slot18Details: <Buffer >, Slot18AdvancedDetails: <Buffer >, Slot19Name: '', Slot19Rasse: '', Slot19Details: <Buffer >, Slot19AdvancedDetails: <Buffer >, Slot20Name: '', Slot20Rasse: '', Slot20Details: <Buffer >, Slot20AdvancedDetails: <Buffer >, Slot21Name: '', Slot21Rasse: '', Slot21Details: <Buffer >, Slot21AdvancedDetails: <Buffer >, Slot22Name: '', Slot22Rasse: '', Slot22Details: <Buffer >, Slot22AdvancedDetails: <Buffer >, Slot23Name: '', Slot23Rasse: '', Slot23Details: <Buffer >, Slot23AdvancedDetails: <Buffer >, Slot24Name: '', Slot24Rasse: '', Slot24Details: <Buffer >, Slot24AdvancedDetails: <Buffer > }

Where for example, Slot1Details's buffer is:

{ team: 0, color_html: '#86E573', color: 'green', apm: 145.81551618814905, actions: 2387, hotkeys: { '0': { assigned: 48, last_totalitems: 5, used: 181 }, '1': { assigned: 17, last_totalitems: 3, used: 122 }, '2': { assigned: 1, last_totalitems: 1, used: 95 }, '3': { assigned: 1, last_totalitems: 2, used: 29 }, '4': { assigned: 1, last_totalitems: 1, used: 107 }, '5': { assigned: 1, last_totalitems: 1, used: 86 } } }

And Slot1AdvancedDetails:

{ actions_details:{ 'Select / deselect':427, 'Right click':858, 'Build / train':55, 'Enter build submenu':14, 'Use ability':131, 'Assign group hotkey':69, 'Select group hotkey':620, 'Select subgroup':49, 'Enter hero\'s abilities submenu':7, 'Basic commands':155, 'Give item / drop item':2 }, units:{ order:{ '2200':'1 Peasant', '2400':'1 Peasant', '30100':'1 Peasant', '41100':'1 Peasant', '66100':'1 Peasant', '66200':'1 Peasant', '66400':'1 Peasant', '94300':'1 Peasant', '95900':'1 Footman', '113600':'1 Footman', '137500':'1 Footman', '192700':'1 Peasant', '192800':'1 Peasant', '274100':'1 Footman', '335000':'1 Footman', '355000':'1 Peasant', '355100':'1 Peasant', '417400':'2 Sorceress', '419300':'1 Peasant', '419400':'1 Peasant', '429800':'1 Sorceress', '430900':'1 Sorceress', '466400':'1 Priest', '489500':'2 Priest', '548500':'2 Priest', '563500':'2 Priest', '619200':'2 Priest', '649100':'1 Peasant', '649200':'1 Peasant', '655800':'1 Sorceress', '727000':'2 Priest', '727800':'1 Footman', '741500':'2 Sorceress', '758700':'1 Footman', '810700':'1 Footman', '846500':'1 Footman', '847200':'2 Priest', '860700':'2 Sorceress', '880700':'2 Sorceress', '904500':'2 Sorceress' }, Peasant:16, Footman:9, Sorceress:13, Priest:13 }, buildings:{ order:{ '10300':'Altar of Kings', '34600':'Barracks', '48600':'Farm', '64100':'Farm', '127900':'Farm', '231200':'Farm', '234000':'Keep', '236800':'Farm', '336400':'Farm', '357400':'Arcane Sanctum', '359600':'Arcane Sanctum', '490600':'Farm', '644700':'Farm', '811900':'Farm' }, 'Altar of Kings':1, Barracks:1, Farm:9, Keep:1, 'Arcane Sanctum':2 }, heroes:{ order:{ '84200':'Archmage', '84400':'Archmage', '355600':'Mountain King' }, Archmage:{ revivals:2, retraining_time:0, ability_time:841300, level:4, abilities:[ Object ] }, 'Mountain King':{ revivals:1, retraining_time:0, ability_time:842700, level:3, abilities:[ Object ] } }, upgrades:{ order:{ '465500':'Sorceress Training', '489200':'Priest Training' }, 'Sorceress Training':1, 'Priest Training':1 }, items:null, hotkeys:{ '0':{ assigned:48, last_totalitems:5, used:181 }, '1':{ assigned:17, last_totalitems:3, used:122 }, '2':{ assigned:1, last_totalitems:1, used:95 }, '3':{ assigned:1, last_totalitems:2, used:29 }, '4':{ assigned:1, last_totalitems:1, used:107 }, '5':{ assigned:1, last_totalitems:1, used:86 } } }

Sorry for the format, use https://jsonformatter.curiousconcept.com to have a better viewing.

[payamazadi]

The in game team ID is useful for me. I have a program that is pulling out the team IDs from my LastReplay file, and using it to calculate who won. It then keeps tally and sends a summary to a Discord channel.

What I'd really love to see added is the game score calculator, like, top total score etc. I can't figure out how to calculate that based on the info output here.

[PBug90]

If total score refers to information displayed on the "after-game-summary"-screen in the game with gold mined etc., you are out of luck. Replays do not contain that information and you need the game engine to precisely re-simulate the inputs made by the players that are stored in the replay files to end up with those statistics.

[PBug90]

Going to close this issue for now. Will create a new issue for an upcoming version 3 soon.