some more infos/flags in fast header parser and AI compatibility

[Skazu]

Hey, we're just making our way to the fast header parser. I have added some flags that we need, is it possible to include them upstream?

Attention: I dived into the binary data a bit and noticed that there seems to be a discrepancy in the player data (where I skip 42 bytes), I thought it was due to RoR, but it seems to be more of a difference between rated and non-rated matches, at least with the skip I can only parse non-rated matches and without the skip only rated matches.

So the PR shouldn't be merged like that, maybe you have an idea how to solve the problem?

[happyleavesaoc]

Happy to merge updates to the fast parser.

I'm not sure about the 42 bytes after the players. I'm currently parsing rated and unrated matches successfully with 1.8.20 (latest version). I am seeing a few parse failures, I can check if it is these bytes and go from there.

[happyleavesaoc]

I forgot to mention, 1.8.20 should already have complete RoR functionality including reference data.

[Skazu]

You are right, it seems to have nothing to do with rated or unrated.

But e.g. this record is parseable with the skip, but not without: https://aoe.ms/replay/?gameId=235478040&profileId=720137

I had a few more but i have no clue whats the reason behind those bytes. And i can't see any dependence on any game settings.

[happyleavesaoc]

FWIW, it's the 42 bytes before it reads the 26 for version 37. You could do a lookahead and optionally skip those bytes if all else fails.

[Skazu]

@happyleavesaoc Current status: I am parsing some additional flags and have made the fast parser "AI compatible" in the sense that by default the AI part is skipped if present. (As was already the case in the Full Parser).

Still not all savegames can be parsed, we still have two typical error cases:

• Custom Scenarios
  • Example:
  • https://www.aoe2insights.com/match/236712984/#overview
• UTF-8 characters in the player name (here I am unsure, but a common characteristic of this kind of error is that there is at least one player with chinese characters in the name)
  • Examples:
  • https://www.aoe2insights.com/match/235469630/#savegames
  • https://www.aoe2insights.com/match/236753758/#savegames
  • https://www.aoe2insights.com/match/236524312/#savegames
  • https://www.aoe2insights.com/match/236726235/#savegames

All these matches cannot be parsed even with the unmodified version, so in my opinion the status is not getting worse.

We have been using this version live for two days now and have already parsed thousands of matches.

I would continue to work on the open error cases separately and submit this status now for the merge, if you still have suggestions for improvement, please give me feedback.

Best regards!

[happyleavesaoc]

Great, I will merge and bump the version then. Thanks!