Open utterances-bot opened 11 months ago
Hey, Den. First of all, thanks for documenting all your work on the Halo Infinite API. I leaned on it quite a bit as I was developing this project. I was curious if you ever had any luck dissecting the film data referenced in the /spectate
response? I've downloaded and decompressed chunks for a sample match, but haven't been able to extract any useful information.
@acurtis166 I am working through that in my free time. It's for sure not anything that is related to video content - it's very likely those are data batches fed to the game engine. I wrote a basic parser that goes through the content, but nothing to share yet.
Huge shoutout to both of you guys! I created a discord bot that generates graphs out of the stats pulled from halo using @acurtis166 's SPNKr.py. There's no public repo yet but the API is available at https://halofin.land/docs.
Now I started wondering if the theater recordings could be used to create heatmaps of the player positions during the match and I was so happy to find both of you my heroes talking here about the theater recordings. Are they encrypted or in human readable form?
Are they encrypted or in human readable form?
It doesn't look like they are encrypted (if they would be - I wouldn't even bother touching it 😄). The challenge there though is that from my cursory analysis, the data captured there is effectively game engine metadata, which means that reading it without understanding the engine might be more complicated than just reading "Position [X, Y, Z] at time [HH:MM:SS]". That being said, I am actively investigating this!
@aapokaapo Exactly what I was hoping for! Halo 5 public API has an endpoint that returns detailed kill events for a given match which was fun to play with.
I was poking around decompressed recording file from the endpoint. Didn't make much procress since I am highly inexperienced in this, but all I got out is that the file consists of 1033 ComponentObjects
such as PlayerWaypointComponent
, ManagedObjectNavpointComponent
and WeaponStateAmmoComponent
to name a few (these components are from the Game Engine I presume). the Component names in the header are 256 bytes long followed by a value or number of somesorts that is 4 bytes long.
I wonder if Halo Forge could help in guessing what a ComponentObject
consists of. Looking at Unity or Unreal gamefile structures could probably help too, since I doubt that the Slipspace engine differs that much from them.
I was poking around decompressed recording file from the endpoint. Didn't make much procress since I am highly inexperienced in this, but all I got out is that the file consists of 1033
ComponentObjects
such asPlayerWaypointComponent
,ManagedObjectNavpointComponent
andWeaponStateAmmoComponent
to name a few (these components are from the Game Engine I presume). the Component names in the header are 256 bytes long followed by a value or number of somesorts that is 4 bytes long.I wonder if Halo Forge could help in guessing what a
ComponentObject
consists of. Looking at Unity or Unreal gamefile structures could probably help too, since I doubt that the Slipspace engine differs that much from them.
What's your approach to getting component names, @aapokaapo? For example, ComponentObject
or PlayetWaypointComponent
.
@dend I opened the file in hex editor and there are 1033 string names readable. I am by no means experienced in this and could be completely in the wrong direction tho...
Thank you so much for documenting your experience exploring the Halo Infinite API. I would like to ask if you can provide any insight regarding the paths that are provided within some of the API responses.
In the Match Progression API call you have shown in this blog post part of the response includes strings labeled “Path.” I was wondering if you could share any insight whatsoever regarding how to use these paths.
https://halostats.svc.halowaypoint.com/hi/players/{player}/matches/{matchId}/progression
{
”ChallengeProgressState”: [
{
"Path": "ChallengeContent/ClientChallengeDefinitions/S1RotationalSet1Challenges/Normal/NTeamSlayerPlay.json"
}
]
}
ChallengeContent/ClientChallengeDefinitions/S1RotationalSet1Challenges/Normal/NTeamSlayerPlay.json
I get the feeling like paths like that include paths to resources like Rank Images or the Player Banners. Resources like that are ultimately what I’m after.
Any advice or response is greatly appreciated! Thank you.
@jackphillips31 you should check out a python lib called SPNKr.py, which is based on this blog
@aapokaapo Thank you for this! I'm creating a React-Native app so I won't actually be able to use this but it has definitely provided some useful information. As far as retrieving images goes, I feel stupid now. I'm not sure how I overlooked the GameCms_GetImage
endpoint, but I did. As far as retrieving retrieving information from paths like ChallengeContent/ClientChallengeDefinitions/S1RotationalSet1Challenges/Normal/NTeamSlayerPlay.json
I'm still not sure. There is a GameCms_GetItem
endpoint so that might be it but I won't know until I get home tonight. I've only briefly looked at SPNKr.py so I'll definitely spend a little more time going over that and seeing if I can gleam anything else from it.
Thank you again!
@jackphillips31 Have a look at the following example.
https://gamecms-hacs.svc.halowaypoint.com/hi/Progression/file/ChallengeContent/ClientChallengeDefinitions/S6CapstoneChallenges/CMedalStick.json
Resulting in the response below.
{
"Description": "Stick an Enemy Spartan with a Grenade",
"Difficulty": "mythic",
"Category": "General",
"Reward": {
"InventoryItems": [],
"OperationExperience": 500,
"InventoryRewards": [
{
"InventoryItemPath": "Inventory/armor/coatings/3761780-ArmorCoating-Mark-VII.json",
"Amount": 1,
"Type": "ArmorCoating"
}
]
},
"ThresholdForSuccess": 1,
"Title": "Mythic Stick"
}
I was getting 404 when trying for your referenced challenge, but the above worked for me (spartan token authentication required). You can see it is relative to the "Progression/file" sub-path within Halo Infinite CMS.
You'll find the relative paths for all career ranks here:
https://gamecms-hacs.svc.halowaypoint.com/hi/Progression/file/RewardTracks/CareerRanks/careerRank1.json
From which you can construct the url to the actual images (example for "Hero" rank below).
https://gamecms-hacs.svc.halowaypoint.com/hi/Images/file/career_rank/NameplateAdornment/272_Hero.png
If instead you are looking for CSR images, try using the following from waypoint (no auth required).
https://www.halowaypoint.com/images/halo-infinite/csr/platinum_5.png
First, get the appearance info for a player:
https://economy.svc.halowaypoint.com/hi/players/{xuid}/customization/appearance
From there you can build the url to retrieve data for a given customization item (backdrop example below).
https://gamecms-hacs.svc.halowaypoint.com/hi/Progression/file/inventory/spartan/backdropimages/103-000-ui-s2-backgro-5c565f83.json
From there you can build the image url:
https://gamecms-hacs.svc.halowaypoint.com/hi/images/file/progression/backgrounds/ui_s2_background_entrenchedfractureinsignia.png
Hope this was helpful
@acurtis166
As I have delved more into the API a lot has become more obvious. At this point I am able to retrieve practically any JSON or any image/image-blob I see. At least I can't think of a single one that I can't retrieve. Except for the example I gave previously which is weird, maybe just a typo? It's unimportant because when I make the same request that resulted in that JSON, the JSON paths in that response I can easily retrieve. It is a little frustrating for something as simple as rebuilding the player banner, you have to make approximately 7 requests just to get the backdrop, banner, and emblem images but I digress.
I'm wondering if there is a dictionary of sorts anywhere where I can gleam information for things like "TeamId"
in the response from Stats_GetMatchStats
endpoint. Like how "TeamId": 0
is Eagle or Red team and how "TeamId": 1
is Cobra or Blue team. Or how "Outcome": 2
is winner and "Outcome": 3
is loser.
that sort of stuff, I'm going to have to compile manually. But if there is a way to get definitions for those, please let me know. I know the xml format is more verbose and in some situations (I believe?) it provides strings rather than numbers but as far as for the Stats_GetMatchStats
endpoint, it seems to provide numbers for the TeamId and Outcome as well.
@acurtis166 I cannot begin to express my appreciation for this, thank you so much! This is an absolutely fantastic resource. I assume you compiled this yourself? Absolutely great work.
@acurtis166 I've made a ton of progress on my personal lil project. But I'm getting stuck again, I'll just get straight to the point to see if you have any insights.
My question is regarding the player's banner/nameplate. I know how to get the backdrop, I know how to get the emblem, I even know how to get the "nameplate adornment" rank images. I'd assume it has something to do with the emblem since in Halo Infinite the emblem and nameplate are one thing sort of? But in the response for the emblem, I'm not seeing anything that is apparently the nameplate.
https://economy.svc.halowaypoint.com/hi/players/{{xuid}}/customization/appearance
{
"Status": "Success",
"Appearance": {
"LastModifiedDateUtc": {
"ISO8601Date": "2024-04-02T00:10:15.85Z"
},
"ActionPosePath": "Inventory/Spartan/ActionPoses/101-000-menu-stance-r-e47e42c4.json",
"BackdropImagePath": "Inventory/Spartan/BackdropImages/103-000-ui-background-e5fd3381.json",
"Emblem": {
"EmblemPath": "Inventory/Spartan/Emblems/104-001-career-rank-0-1e8f3859.json",
"ConfigurationId": -410600460
},
"ServiceTag": "MVP",
"IntroEmotePath": null,
"PlayerTitlePath": null
}
}
https://gamecms-hacs.svc.halowaypoint.com:443/hi/Progression/file/Inventory/Spartan/Emblems/104-001-olympus-skull-2845463e.json
{
"TagId": 0,
"ThemeName": {
"m_identifier": 626314953
},
"EmblemShaderName": {
"m_identifier": -1
},
"AvailableConfigurations": [
{
"ConfigurationId": -2101285248,
"ConfigurationPath": "Configuration/Emblems/Coatings/1000-000-2c95b554.json"
},
{
"ConfigurationId": -1300612690,
"ConfigurationPath": "Configuration/Emblems/Coatings/1000-000-f37482bf.json"
},
{
"ConfigurationId": 2092759023,
"ConfigurationPath": "Configuration/Emblems/Coatings/1000-000-f9d7584d.json"
},
{
"ConfigurationId": -1756788076,
"ConfigurationPath": "Configuration/Emblems/Coatings/1000-000-1de32d39.json"
},
{
"ConfigurationId": 2029654151,
"ConfigurationPath": "Configuration/Emblems/Coatings/1000-000-715a8962.json"
}
],
"CommonData": {
"Id": "104-001-olympus-skull-2845463e",
"HideUntilOwned": false,
"Title": {
"status": "Ready",
"value": "Skull King",
"translations": {
"cs-CZ": "Král lebek",
"da-DK": "Kraniekonge",
"de-DE": "Schädelkönig",
"el-GR": "Βασιλιάς των κρανίων",
"es-ES": "Rey de las calaveras",
"es-MX": "Rey de los cráneos",
"fi-FI": "Kallokuningas",
"fr-FR": "Crâne couronné",
"hu-HU": "Koponyakirály",
"it-IT": "Re teschio",
"ja-JP": "スカル キング",
"ko-KR": "해골 왕",
"nb-NO": "Skallekonge",
"nl-NL": "Schedelkoning",
"pl-PL": "Król czaszek",
"pt-BR": "Rei da Caveira",
"pt-PT": "Rei Caveira",
"ru-RU": "Король черепов",
"sv-SE": "Skallkung",
"tr-TR": "Kafatası Kralı",
"zh-CN": "骷髅王",
"zh-TW": "骷髏王",
"qps-ploc": "Ѕќůłĺ Кïπĝ !!!",
"qps-ploca": "Šкϋľľ Κіňğ !!!",
"qps-plocm": "Ŝĸυļℓ Кΐηĝ !!!"
}
},
"Description": {
"status": "Ready",
"value": "Lord of hill and dale.",
"translations": {
"cs-CZ": "Pán kopců a údolí.",
"da-DK": "Herre over bakke og dal.",
"de-DE": "Herr über Berg und Tal.",
"el-GR": "Άρχοντας του λόφου και της κοιλάδας.",
"es-ES": "Señor de la colina y el valle.",
"es-MX": "Señor de la colina y el valle.",
"fi-FI": "Kukkulan ja laakson herra.",
"fr-FR": "Seigneur de la colline et du vallon.",
"hu-HU": "Dombok-völgyek ura.",
"it-IT": "Signore della collina e della valle.",
"ja-JP": "丘と谷の王。",
"ko-KR": "울퉁불퉁한 땅의 주인입니다.",
"nb-NO": "Hersker over fjell og daler.",
"nl-NL": "Heer van heuvel en dal.",
"pl-PL": "Pan wzgórz i dolin.",
"pt-BR": "Senhor de colinas e vales.",
"pt-PT": "Senhor da colina e do vale.",
"ru-RU": "Владыка холмов и долин.",
"sv-SE": "Härskare över berg och dal.",
"tr-TR": "Dere tepenin efendisi.",
"zh-CN": "山谷之主。",
"zh-TW": "山丘之主。",
"qps-ploc": "Łōгď öƒ ĥĩļł ãňď ðдŀė. !!! !!! ",
"qps-ploca": "£σяδ бƒ ĥĭļł âπð đąľе. !!! !!! ",
"qps-plocm": "£οřð бƒ нĩļł аņδ ďåļё. !!! !!! "
}
},
"FeatureFlag": true,
"ItemAvailability": {
"status": "Test",
"value": "",
"translations": {}
},
"DateReleased": {
"ISO8601Date": ""
},
"AltName": {
"status": "Test",
"value": "104-001-olympus-skull-2845463e",
"translations": {
"qps-ploc": "104-001-òŀумφúš-śκΰłŀ-2845463é !!! !!! !!!",
"qps-ploca": "104-001-ŏŀýmφųѕ-ŝкūłℓ-2845463é !!! !!! !!!",
"qps-plocm": "104-001-őļўmρΰѕ-ŝķũŀł-2845463з !!! !!! !!!"
}
},
"IconStringId": {
"m_identifier": -1
},
"SpriteBitmap": 0,
"SpriteFrameIndex": 0,
"AltSpriteBitmap": 0,
"AltSpriteFrameIndex": 0,
"DisplayPath": {
"Width": 585,
"Height": 802,
"Media": {
"MediaUrl": {
"AuthorityId": "",
"Path": "progression/Inventory/Emblems/olympus_SkullKing_emblem.png",
"RetryPolicyId": "",
"TopicName": "",
"AcknowledgementTypeId": "NoAcknowledgement",
"AuthenticationLifetimeExtensionSupported": false,
"ClearanceAware": false
},
"MimeType": "",
"Caption": {
"status": "Test",
"value": "",
"translations": {}
},
"AlternateText": {
"status": "Test",
"value": "",
"translations": {}
},
"FolderPath": "",
"FileName": ""
},
"MimeType": "image/png",
"FolderPath": "progression/Inventory/Emblems",
"FileName": "olympus_SkullKing_emblem.png"
},
"Quality": "Common",
"ManufacturerId": 26,
"Type": "SpartanEmblem",
"RewardTrack": "",
"ParentPaths": [],
"SortingMetadata": {
"categoryWeight": 0,
"subCategoryWeight": 0
},
"SeasonNumber": 1,
"OriginalSeasonNumber": 1,
"IsCrossCompatible": true,
"Season": {
"status": "Test",
"value": "",
"translations": {}
}
}
}
I assume the configurations have to do with the color palette but I tried those and they don't have any useful information either. On the note of the configurations, I'm also wondering how to get the different color emblems/nameplates.
{
"ConfigurationId": {
"m_identifier": -1300612690
},
"CommonData": {
"Id": "1000-000-f37482bf",
"HideUntilOwned": false,
"Title": {
"status": "Frozen",
"value": "Color Palette",
"translations": {}
},
"Description": {
"status": "Frozen",
"value": "0",
"translations": {
"de-DE": "343other_adrenaline",
"es-ES": "343other_adrenaline",
"es-MX": "343other_adrenaline",
"zh-CN": "343other_adrenaline",
"zh-TW": "343other_adrenaline",
"qps-ploc": "343őťћęґ_ăďгэñáĺĩπê !!! !!!",
"qps-ploca": "343бτĥ℮ř_ªđŗëńăĺĭπё !!! !!!",
"qps-plocm": "343σŧĥèя_äđřęŋаℓīиè !!! !!!"
}
},
"FeatureFlag": true,
"ItemAvailability": {
"status": "Test",
"value": "",
"translations": {}
},
"DateReleased": {
"ISO8601Date": ""
},
"AltName": {
"status": "Test",
"value": "",
"translations": {}
},
"IconStringId": {
"m_identifier": -1
},
"SpriteBitmap": 0,
"SpriteFrameIndex": 0,
"AltSpriteBitmap": 0,
"AltSpriteFrameIndex": 0,
"DisplayPath": {
"Width": 355,
"Height": 327,
"Media": {
"MediaUrl": {
"AuthorityId": "",
"Path": "progression/Inventory/Emblems/emblem_palette.PNG",
"RetryPolicyId": "",
"TopicName": "",
"AcknowledgementTypeId": "NoAcknowledgement",
"AuthenticationLifetimeExtensionSupported": false,
"ClearanceAware": false
},
"MimeType": "",
"Caption": {
"status": "Test",
"value": "",
"translations": {}
},
"AlternateText": {
"status": "Test",
"value": "",
"translations": {}
},
"FolderPath": "",
"FileName": ""
},
"MimeType": "image/png",
"FolderPath": "progression/Inventory/Emblems",
"FileName": "emblem_palette.PNG"
},
"Quality": "Common",
"ManufacturerId": 0,
"Type": "None",
"RewardTrack": "",
"ParentPaths": [],
"SortingMetadata": {
"categoryWeight": 0,
"subCategoryWeight": 0
},
"SeasonNumber": 1,
"OriginalSeasonNumber": 1,
"IsCrossCompatible": false,
"Season": {
"status": "Frozen",
"value": "",
"translations": {}
}
}
}
@jackphillips31 @acurtis166 Actually while I was writing that post I found a kind of hacky way to do it. I wanted to post it anyways just to answer it myself lol.
I found this solution while looking at the Halo Waypoint website trying to figure out how they're getting their nameplate image. If you copy the image URL though it'll give you a link like this: blob:https://www.halowaypoint.com/8144fcc0-ea16-4db0-a68c-03b3b96c888d
and this link ONLY works while you have that session open.
I wasn't hopeful when inspecting the network traffic because I assumed all the requests would be done server-side. But lo and behold... there it was! This endpoint only requires your Spartan Token. Without further adoe, here are the endpoints...
Endpoint for Emblems:
https://gamecms-hacs.svc.halowaypoint.com/hi/Waypoint/file/images/emblems
Endpoint for nameplates:
https://gamecms-hacs.svc.halowaypoint.com/hi/Waypoint/file/images/nameplates
Simple. When you make your Economy_PlayerCustomizationAppearance API call, you should get a response that looks something like this:
{
"Status": "Success",
"Appearance": {
"LastModifiedDateUtc": {
"ISO8601Date": "2024-03-11T22:17:53.733Z"
},
"ActionPosePath": "Inventory/Spartan/ActionPoses/101-000-menu-stance-r-e47e42c4.json",
"BackdropImagePath": "Inventory/Spartan/BackdropImages/103-000-ui-background-e5fd3381.json",
"Emblem": {
"EmblemPath": "Inventory/Spartan/Emblems/104-001-olympus-skull-2845463e.json",
"ConfigurationId": -2101285248
},
"ServiceTag": "MVP",
"IntroEmotePath": null,
"PlayerTitlePath": null
}
}
You're going to want to take note of both the ConfigurationId and the EmblemPath From here you're going to want to use the GameCms_GetItem API call with the EmblemPath. The response you get from that is a little too big to post so I'll cut out what isn't important but it should look like this:
{
"TagId": 0,
"ThemeName": {
"m_identifier": 626314953
},
"EmblemShaderName": {
"m_identifier": -1
},
...
"CommonData": {
...
"AltName": {
"status": "Test",
"value": "104-001-olympus-skull-2845463e",
"translations": {
"qps-ploc": "104-001-òŀумφúš-śκΰłŀ-2845463é !!! !!! !!!",
"qps-ploca": "104-001-ŏŀýmφųѕ-ŝкūłℓ-2845463é !!! !!! !!!",
"qps-plocm": "104-001-őļўmρΰѕ-ŝķũŀł-2845463з !!! !!! !!!"
}
},
...
}
}
What is important here is the response.CommonData.AltName.value
in this case being 104-001-olympus-skull-2845463e. Bare with me, we are almost there. This is where the ConfigurationId comes into play. You're going to want to slap on the ConfigurationId onto the back of the AltName string.
If your ConfigurationId is negative then replace the - sign with an n. In my example the ConfigurationId is -2101285248 so we would be adding n2101285248 to the end of the AltName string. Mind you there will be an underscore inbetween and of course we will also be adding a .png to the end as well.
So our finished filename will look like this: 104-001-olympus-skull-2845463e_n2101285248.png Using the endpoints above along with our Spartan Token, we can now retrieve the colored versions of both our emblems and nameplates.
Emblem URL:
https://gamecms-hacs.svc.halowaypoint.com/hi/Waypoint/file/images/emblems/104-001-olympus-skull-2845463e_n2101285248.png
Nameplate URL:
https://gamecms-hacs.svc.halowaypoint.com/hi/Waypoint/file/images/nameplates/104-001-olympus-skull-2845463e_n2101285248.png
@jackphillips31 Nice! I haven't added any of the economy endpoints so this will be helpful when I get to that. Thanks for sharing!
@acurtis166 I actually just found an endpoint that gives the paths to all the emblems/nameplates and the text color for their corresponding configuration!
The endpoint:
https://gamecms-hacs.svc.halowaypoint.com/hi/Waypoint/file/images/emblems/mapping.json
The response has MANY more items than this but it looks like:
{
"104-001-olympus-angry-d0fe80dc":{
"-1408018525":{
"emblemCmsPath":"images/emblems/104-001-olympus-angry-d0fe80dc_n1408018525.png",
"nameplateCmsPath":"images/nameplates/104-001-olympus-angry-d0fe80dc_n1408018525.png",
"textColor":"#000000"
},
"-711804461":{
"emblemCmsPath":"images/emblems/104-001-olympus-angry-d0fe80dc_n711804461.png",
"nameplateCmsPath":"images/nameplates/104-001-olympus-angry-d0fe80dc_n711804461.png",
"textColor":"#FFFFFF"
},
"-1490538315":{
"emblemCmsPath":"images/emblems/104-001-olympus-angry-d0fe80dc_n1490538315.png",
"nameplateCmsPath":"images/nameplates/104-001-olympus-angry-d0fe80dc_n1490538315.png",
"textColor":"#000000"
},
"-1561561067":{
"emblemCmsPath":"images/emblems/104-001-olympus-angry-d0fe80dc_n1561561067.png",
"nameplateCmsPath":"images/nameplates/104-001-olympus-angry-d0fe80dc_n1561561067.png",
"textColor":"#FFFFFF"
},
"1740959434":{
"emblemCmsPath":"images/emblems/104-001-olympus-angry-d0fe80dc_1740959434.png",
"nameplateCmsPath":"images/nameplates/104-001-olympus-angry-d0fe80dc_1740959434.png",
"textColor":"#000000"
},
"1387731827":{
"emblemCmsPath":"images/emblems/104-001-olympus-angry-d0fe80dc_1387731827.png",
"nameplateCmsPath":"images/nameplates/104-001-olympus-angry-d0fe80dc_1387731827.png",
"textColor":"#FFFFFF"
},
"-1788399426":{
"emblemCmsPath":"images/emblems/104-001-olympus-angry-d0fe80dc_n1788399426.png",
"nameplateCmsPath":"images/nameplates/104-001-olympus-angry-d0fe80dc_n1788399426.png",
"textColor":"#000000"
}
},
}
@acurtis166 @jackphillips31 you might also find this relevant: https://den.dev/blog/openspartan-battlepass/#a-horse-with-no-nameplate
I've been tackling the issue of "missing" nameplates and player emblems, so you might want to familiarize yourself with the exceptions 😊 (the ranked banners are my favorite, but are not available in the API).
Hey guys have you seen this tool: https://github.com/Gamergotten/Infinite-runtime-tagviewer I wonder if this could be used to crack the theater recordings open
@aapokaapo haven't looked at it yet, but these past few evenings I've been digging through the film files anyway 😊
@aapokaapo I sent the bin files over to a few awesome Modders I know that are very familiar with this tool. I will report back if anyone cracks the code. I want heatmaps something terrible!
@dend @aapokaapo @ChaseWoodhams Not sure if you guys have had any updates on the film data, but I wanted to share some of my results from looking at the highlight events chunk type files in case any of you are able to do something fun with it.
So the highlight event chunk is the last of the chunks referenced by the metadata response and contains the events referenced in the timeline when viewing a match in theater mode.
I was able to parse out these events by finding 64-bit XUIDs preceding specific bytes and then interpreting the binary content that followed. JSON representation of a parsed event:
{
"xuid": 1234567890123456,
"gamertag": "Gamer123",
"type_hint": 50,
"is_medal": true,
"event_type": "medal",
"time_ms": 341426,
"medal_value": 71,
"medal_name": "Rifleman"
}
Since then I've been playing with how to use the data and have produced a couple interesting plots.
Plot displaying players' KD-difference over the duration of the match:
Plot showing killer-victim counts (involves a join between kill events and death events using a 5 ms tolerance):
Unfortunately, I haven't found any indication of player position or weapon usage, though I'm not expecting to given the events seem to just be used for navigation. I'd still like to see if that information can be pulled from the replication data chunks.
If you are interested in the code, it can be found here.
Here's a minimal example using spnkr
.
import asyncio
import json
from aiohttp import ClientSession
from spnkr import HaloInfiniteClient, film
async def main():
async with ClientSession() as session:
client = HaloInfiniteClient(
session, spartan_token="SPARTAN_TOKEN", clearance_token=""
)
events = await film.read_highlight_events(client, match_id="MATCH_GUID")
with open("highlight_events.json", "w") as fp:
json.dump([e._asdict() for e in events], fp)
if __name__ == "__main__":
asyncio.run(main())
Amazing work!
I had pretty much given up with this, since cracking the code was way out of my skills.
Awesome job @acurtis166. The more I dig through it the more I am convinced that the positioning data is for the engine rather some kind of direct coordinates on the map. I am trying to replicate that with a custom game to see how values change over the same map.
@acurtis166 I was doing some tests and seems like the "last chunk" analysis could be significantly flawed. For example, take this match film (one of my Husky Raid games):
https://discovery-infiniteugc.svc.halowaypoint.com
/hi
/films
/matches
/4fb89c93-53e1-4d7e-b273-5f4c4c1a58e4
/spectate
Looking at the overall game performance in the PGCR:
Quick stats:
However, looking at /filmChunk22
alone, specifically for my gamertag, I only see 10 occurrences:
While you are right that the last chunk is the chunk that logs all events, we should check if these events are not grouped or otherwise compressed into the player data. Will be digging through it more.
@dend Yes! That held me up for a long time, but then I realized the events aren't byte-aligned.
I'm sure there are plenty of tools for this, but I used the Python package bitstring:
import zlib
import bitstring
file = ... # Path to compressed highlight events chunk
byte_aligned = ... # True | False
with open(file, "rb") as fp:
data = zlib.decompress(fp.read())
bits = bitstring.Bits(bytes=data)
term = "ZeBond".encode("utf-16le")
n = 0
for start in bits.findall(term, bytealigned=byte_aligned):
remainder = start % 8
hexstr = bits[start : start + (60 * 8)].tobytes().hex(" ")
message = f"start={start:<8} | remainder={remainder} | hex={hexstr}"
print(message)
n += 1
print(n, "found")
With byte_aligned = True
:
start=270496 | remainder=0 | hex=5a 00 65 00 42 00 6f 00 6e 00 64 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 03 01 01 00 00 00 00 00 00 00 00 00 00 00 00 32 00 00 b4 b6 00 00 00 00 00 00 00 00
start=376440 | remainder=0 | hex=5a 00 65 00 42 00 6f 00 6e 00 64 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 03 01 01 00 00 00 00 00 00 00 00 00 00 00 00 dc 00 00 c2 54 00 00 00 01 00 00 00 02
start=588456 | remainder=0 | hex=5a 00 65 00 42 00 6f 00 6e 00 64 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 03 01 01 00 00 00 00 00 00 00 00 00 00 00 00 14 00 00 ce d8 00 00 00 00 00 00 00 01
start=2931864 | remainder=0 | hex=5a 00 65 00 42 00 6f 00 6e 00 64 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 03 01 01 00 00 00 00 00 00 00 00 00 00 00 00 0a 00 02 d3 97 00 00 00 00 00 00 00 02
start=3652600 | remainder=0 | hex=5a 00 65 00 42 00 6f 00 6e 00 64 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 03 01 01 00 00 00 00 00 00 00 00 00 00 00 00 64 00 03 55 04 00 00 00 01 00 00 00 00
start=4820824 | remainder=0 | hex=5a 00 65 00 42 00 6f 00 6e 00 64 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 03 01 01 00 00 00 00 00 00 00 00 00 00 00 00 32 00 04 32 a9 00 00 00 01 00 00 00 4b
start=4997456 | remainder=0 | hex=5a 00 65 00 42 00 6f 00 6e 00 64 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 03 01 01 00 00 00 00 00 00 00 00 00 00 00 00 64 00 04 4e 29 00 00 00 01 00 00 00 63
start=5224664 | remainder=0 | hex=5a 00 65 00 42 00 6f 00 6e 00 64 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 03 01 01 00 00 00 00 00 00 00 00 00 00 00 00 32 00 04 6a 61 00 00 00 00 00 00 00 00
start=5648592 | remainder=0 | hex=5a 00 65 00 42 00 6f 00 6e 00 64 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 03 01 01 00 00 00 00 00 00 00 00 00 00 00 00 14 00 04 e6 a9 00 00 00 00 00 00 00 01
start=7194120 | remainder=0 | hex=5a 00 65 00 42 00 6f 00 6e 00 64 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 03 01 01 00 00 00 00 00 00 00 00 00 00 00 00 96 00 05 f1 c8 00 00 00 01 00 00 00 20
10 found
With byte_aligned = False
:
start=23204 | remainder=4 | hex=5a 00 65 00 42 00 6f 00 6e 00 64 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 03 01 01 00 00 00 00 00 00 00 00 00 00 00 00 32 00 00 9e 2a 00 00 00 00 00 00 00 00
start=176269 | remainder=5 | hex=5a 00 65 00 42 00 6f 00 6e 00 64 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 03 01 01 00 00 00 00 00 00 00 00 00 00 00 00 32 00 00 a8 86 00 00 00 01 00 00 00 83
start=199825 | remainder=1 | hex=5a 00 65 00 42 00 6f 00 6e 00 64 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 03 01 01 00 00 00 00 00 00 00 00 00 00 00 00 32 00 00 a8 86 00 00 00 00 00 00 00 00
start=246940 | remainder=4 | hex=5a 00 65 00 42 00 6f 00 6e 00 64 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 03 01 01 00 00 00 00 00 00 00 00 00 00 00 00 64 00 00 a8 87 00 00 00 01 00 00 00 00
start=270496 | remainder=0 | hex=5a 00 65 00 42 00 6f 00 6e 00 64 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 03 01 01 00 00 00 00 00 00 00 00 00 00 00 00 32 00 00 b4 b6 00 00 00 00 00 00 00 00
start=306210 | remainder=2 | hex=5a 00 65 00 42 00 6f 00 6e 00 64 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 03 01 01 00 00 00 00 00 00 00 00 00 00 00 00 96 00 00 b4 b6 00 00 00 01 00 00 00 01
start=329766 | remainder=6 | hex=5a 00 65 00 42 00 6f 00 6e 00 64 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 03 01 01 00 00 00 00 00 00 00 00 00 00 00 00 32 00 00 c2 54 00 00 00 00 00 00 00 00
start=376440 | remainder=0 | hex=5a 00 65 00 42 00 6f 00 6e 00 64 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 03 01 01 00 00 00 00 00 00 00 00 00 00 00 00 dc 00 00 c2 54 00 00 00 01 00 00 00 02
start=447111 | remainder=7 | hex=5a 00 65 00 42 00 6f 00 6e 00 64 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 03 01 01 00 00 00 00 00 00 00 00 00 00 00 00 32 00 00 cc 0b 00 00 00 00 00 00 00 00
start=494226 | remainder=2 | hex=5a 00 65 00 42 00 6f 00 6e 00 64 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 03 01 01 00 00 00 00 00 00 00 00 00 00 00 00 64 00 00 cc 0b 00 00 00 01 00 00 00 09
start=517782 | remainder=6 | hex=5a 00 65 00 42 00 6f 00 6e 00 64 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 03 01 01 00 00 00 00 00 00 00 00 00 00 00 00 e1 00 00 cc 0b 00 00 00 01 00 00 00 03
start=588456 | remainder=0 | hex=5a 00 65 00 42 00 6f 00 6e 00 64 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 03 01 01 00 00 00 00 00 00 00 00 00 00 00 00 14 00 00 ce d8 00 00 00 00 00 00 00 01
start=788198 | remainder=6 | hex=5a 00 65 00 42 00 6f 00 6e 00 64 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 03 01 01 00 00 00 00 00 00 00 00 00 00 00 00 32 00 01 04 8c 00 00 00 00 00 00 00 00
start=941698 | remainder=2 | hex=5a 00 65 00 42 00 6f 00 6e 00 64 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 03 01 01 00 00 00 00 00 00 00 00 00 00 00 00 32 00 01 2c b0 00 00 00 00 00 00 00 00
start=988372 | remainder=4 | hex=5a 00 65 00 42 00 6f 00 6e 00 64 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 03 01 01 00 00 00 00 00 00 00 00 00 00 00 00 32 00 01 2d 15 00 00 00 00 00 00 00 00
start=1024086 | remainder=6 | hex=5a 00 65 00 42 00 6f 00 6e 00 64 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 03 01 01 00 00 00 00 00 00 00 00 00 00 00 00 64 00 01 2d 15 00 00 00 01 00 00 00 00
start=1047642 | remainder=2 | hex=5a 00 65 00 42 00 6f 00 6e 00 64 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 03 01 01 00 00 00 00 00 00 00 00 00 00 00 00 32 00 01 3c 54 00 00 00 00 00 00 00 00
start=1094757 | remainder=5 | hex=5a 00 65 00 42 00 6f 00 6e 00 64 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 03 01 01 00 00 00 00 00 00 00 00 00 00 00 00 96 00 01 3c 55 00 00 00 01 00 00 00 01
start=1164549 | remainder=5 | hex=5a 00 65 00 42 00 6f 00 6e 00 64 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 03 01 01 00 00 00 00 00 00 00 00 00 00 00 00 32 00 01 54 92 00 00 00 00 00 00 00 00
start=1200263 | remainder=7 | hex=5a 00 65 00 42 00 6f 00 6e 00 64 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 03 01 01 00 00 00 00 00 00 00 00 00 00 00 00 64 00 01 54 92 00 00 00 01 00 00 00 09
start=1317611 | remainder=3 | hex=5a 00 65 00 42 00 6f 00 6e 00 64 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 03 01 01 00 00 00 00 00 00 00 00 00 00 00 00 32 00 01 6b 0e 00 00 00 00 00 00 00 00
start=1364726 | remainder=6 | hex=5a 00 65 00 42 00 6f 00 6e 00 64 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 03 01 01 00 00 00 00 00 00 00 00 00 00 00 00 32 00 01 6b 0e 00 00 00 01 00 00 00 4c
start=1388282 | remainder=2 | hex=5a 00 65 00 42 00 6f 00 6e 00 64 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 03 01 01 00 00 00 00 00 00 00 00 00 00 00 00 32 00 01 82 c7 00 00 00 00 00 00 00 00
start=1612778 | remainder=2 | hex=5a 00 65 00 42 00 6f 00 6e 00 64 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 03 01 01 00 00 00 00 00 00 00 00 00 00 00 00 32 00 01 af 4d 00 00 00 00 00 00 00 00
start=1730883 | remainder=3 | hex=5a 00 65 00 42 00 6f 00 6e 00 64 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 03 01 01 00 00 00 00 00 00 00 00 00 00 00 00 0a 00 01 cb 74 00 00 00 00 00 00 00 02
start=1801557 | remainder=5 | hex=5a 00 65 00 42 00 6f 00 6e 00 64 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 03 01 01 00 00 00 00 00 00 00 00 00 00 00 00 14 00 01 d5 09 00 00 00 00 00 00 00 01
start=2048859 | remainder=3 | hex=5a 00 65 00 42 00 6f 00 6e 00 64 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 03 01 01 00 00 00 00 00 00 00 00 00 00 00 00 64 00 02 0f 7d 00 00 00 01 00 00 00 64
start=2072415 | remainder=7 | hex=5a 00 65 00 42 00 6f 00 6e 00 64 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 03 01 01 00 00 00 00 00 00 00 00 00 00 00 00 32 00 02 0f 7e 00 00 00 00 00 00 00 00
start=2119530 | remainder=2 | hex=5a 00 65 00 42 00 6f 00 6e 00 64 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 03 01 01 00 00 00 00 00 00 00 00 00 00 00 00 32 00 02 25 21 00 00 00 00 00 00 00 00
start=2178805 | remainder=5 | hex=5a 00 65 00 42 00 6f 00 6e 00 64 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 03 01 01 00 00 00 00 00 00 00 00 00 00 00 00 14 00 02 27 05 00 00 00 00 00 00 00 01
start=2565699 | remainder=3 | hex=5a 00 65 00 42 00 6f 00 6e 00 64 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 03 01 01 00 00 00 00 00 00 00 00 00 00 00 00 14 00 02 78 39 00 00 00 00 00 00 00 01
start=2625729 | remainder=1 | hex=5a 00 65 00 42 00 6f 00 6e 00 64 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 03 01 01 00 00 00 00 00 00 00 00 00 00 00 00 32 00 02 9c 64 00 00 00 00 00 00 00 00
start=2814513 | remainder=1 | hex=5a 00 65 00 42 00 6f 00 6e 00 64 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 03 01 01 00 00 00 00 00 00 00 00 00 00 00 00 32 00 02 cf 7b 00 00 00 00 00 00 00 00
start=2931864 | remainder=0 | hex=5a 00 65 00 42 00 6f 00 6e 00 64 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 03 01 01 00 00 00 00 00 00 00 00 00 00 00 00 0a 00 02 d3 97 00 00 00 00 00 00 00 02
start=2978981 | remainder=5 | hex=5a 00 65 00 42 00 6f 00 6e 00 64 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 03 01 01 00 00 00 00 00 00 00 00 00 00 00 00 14 00 02 d8 bd 00 00 00 00 00 00 00 01
start=3214131 | remainder=3 | hex=5a 00 65 00 42 00 6f 00 6e 00 64 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 03 01 01 00 00 00 00 00 00 00 00 00 00 00 00 32 00 03 0a 34 00 00 00 00 00 00 00 00
start=3333439 | remainder=7 | hex=5a 00 65 00 42 00 6f 00 6e 00 64 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 03 01 01 00 00 00 00 00 00 00 00 00 00 00 00 14 00 03 0d fc 00 00 00 00 00 00 00 01
start=3369153 | remainder=1 | hex=5a 00 65 00 42 00 6f 00 6e 00 64 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 03 01 01 00 00 00 00 00 00 00 00 00 00 00 00 32 00 03 17 d1 00 00 00 01 00 00 00 5b
start=3534814 | remainder=6 | hex=5a 00 65 00 42 00 6f 00 6e 00 64 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 03 01 01 00 00 00 00 00 00 00 00 00 00 00 00 32 00 03 42 a3 00 00 00 00 00 00 00 00
start=3581929 | remainder=1 | hex=5a 00 65 00 42 00 6f 00 6e 00 64 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 03 01 01 00 00 00 00 00 00 00 00 00 00 00 00 32 00 03 42 a3 00 00 00 01 00 00 00 48
start=3605485 | remainder=5 | hex=5a 00 65 00 42 00 6f 00 6e 00 64 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 03 01 01 00 00 00 00 00 00 00 00 00 00 00 00 32 00 03 55 04 00 00 00 00 00 00 00 00
start=3652600 | remainder=0 | hex=5a 00 65 00 42 00 6f 00 6e 00 64 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 03 01 01 00 00 00 00 00 00 00 00 00 00 00 00 64 00 03 55 04 00 00 00 01 00 00 00 00
start=3699717 | remainder=5 | hex=5a 00 65 00 42 00 6f 00 6e 00 64 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 03 01 01 00 00 00 00 00 00 00 00 00 00 00 00 14 00 03 5c 1f 00 00 00 00 00 00 00 01
start=3899905 | remainder=1 | hex=5a 00 65 00 42 00 6f 00 6e 00 64 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 03 01 01 00 00 00 00 00 00 00 00 00 00 00 00 14 00 03 80 39 00 00 00 00 00 00 00 01
start=4242206 | remainder=6 | hex=5a 00 65 00 42 00 6f 00 6e 00 64 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 03 01 01 00 00 00 00 00 00 00 00 00 00 00 00 0a 00 03 bd 9d 00 00 00 00 00 00 00 02
start=4265762 | remainder=2 | hex=5a 00 65 00 42 00 6f 00 6e 00 64 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 03 01 01 00 00 00 00 00 00 00 00 00 00 00 00 32 00 03 bf 3d 00 00 00 00 00 00 00 00
start=4395714 | remainder=2 | hex=5a 00 65 00 42 00 6f 00 6e 00 64 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 03 01 01 00 00 00 00 00 00 00 00 00 00 00 00 32 00 03 de 75 00 00 00 00 00 00 00 00
start=4513819 | remainder=3 | hex=5a 00 65 00 42 00 6f 00 6e 00 64 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 03 01 01 00 00 00 00 00 00 00 00 00 00 00 00 32 00 04 07 c6 00 00 00 00 00 00 00 00
start=4620212 | remainder=4 | hex=5a 00 65 00 42 00 6f 00 6e 00 64 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 03 01 01 00 00 00 00 00 00 00 00 00 00 00 00 32 00 04 1a 28 00 00 00 00 00 00 00 00
start=4655926 | remainder=6 | hex=5a 00 65 00 42 00 6f 00 6e 00 64 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 03 01 01 00 00 00 00 00 00 00 00 00 00 00 00 64 00 04 1a 28 00 00 00 01 00 00 00 00
start=4679482 | remainder=2 | hex=5a 00 65 00 42 00 6f 00 6e 00 64 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 03 01 01 00 00 00 00 00 00 00 00 00 00 00 00 32 00 04 21 b7 00 00 00 00 00 00 00 00
start=4726597 | remainder=5 | hex=5a 00 65 00 42 00 6f 00 6e 00 64 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 03 01 01 00 00 00 00 00 00 00 00 00 00 00 00 64 00 04 21 b7 00 00 00 01 00 00 00 09
start=4750153 | remainder=1 | hex=5a 00 65 00 42 00 6f 00 6e 00 64 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 03 01 01 00 00 00 00 00 00 00 00 00 00 00 00 96 00 04 21 b7 00 00 00 01 00 00 00 01
start=4773709 | remainder=5 | hex=5a 00 65 00 42 00 6f 00 6e 00 64 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 03 01 01 00 00 00 00 00 00 00 00 00 00 00 00 32 00 04 32 a9 00 00 00 00 00 00 00 00
start=4820824 | remainder=0 | hex=5a 00 65 00 42 00 6f 00 6e 00 64 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 03 01 01 00 00 00 00 00 00 00 00 00 00 00 00 32 00 04 32 a9 00 00 00 01 00 00 00 4b
start=4844380 | remainder=4 | hex=5a 00 65 00 42 00 6f 00 6e 00 64 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 03 01 01 00 00 00 00 00 00 00 00 00 00 00 00 dc 00 04 32 a9 00 00 00 01 00 00 00 02
start=4997456 | remainder=0 | hex=5a 00 65 00 42 00 6f 00 6e 00 64 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 03 01 01 00 00 00 00 00 00 00 00 00 00 00 00 64 00 04 4e 29 00 00 00 01 00 00 00 63
start=5021012 | remainder=4 | hex=5a 00 65 00 42 00 6f 00 6e 00 64 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 03 01 01 00 00 00 00 00 00 00 00 00 00 00 00 32 00 04 4e 29 00 00 00 00 00 00 00 00
start=5224664 | remainder=0 | hex=5a 00 65 00 42 00 6f 00 6e 00 64 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 03 01 01 00 00 00 00 00 00 00 00 00 00 00 00 32 00 04 6a 61 00 00 00 00 00 00 00 00
start=5318899 | remainder=3 | hex=5a 00 65 00 42 00 6f 00 6e 00 64 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 03 01 01 00 00 00 00 00 00 00 00 00 00 00 00 14 00 04 76 7f 00 00 00 00 00 00 00 01
start=5389132 | remainder=4 | hex=5a 00 65 00 42 00 6f 00 6e 00 64 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 03 01 01 00 00 00 00 00 00 00 00 00 00 00 00 32 00 04 a8 07 00 00 00 00 00 00 00 00
start=5507681 | remainder=1 | hex=5a 00 65 00 42 00 6f 00 6e 00 64 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 03 01 01 00 00 00 00 00 00 00 00 00 00 00 00 14 00 04 bf 4d 00 00 00 00 00 00 00 01
start=5648592 | remainder=0 | hex=5a 00 65 00 42 00 6f 00 6e 00 64 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 03 01 01 00 00 00 00 00 00 00 00 00 00 00 00 14 00 04 e6 a9 00 00 00 00 00 00 00 01
start=5920970 | remainder=2 | hex=5a 00 65 00 42 00 6f 00 6e 00 64 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 03 01 01 00 00 00 00 00 00 00 00 00 00 00 00 32 00 05 11 26 00 00 00 00 00 00 00 00
start=6003358 | remainder=6 | hex=5a 00 65 00 42 00 6f 00 6e 00 64 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 03 01 01 00 00 00 00 00 00 00 00 00 00 00 00 0a 00 05 16 1b 00 00 00 00 00 00 00 02
start=6026914 | remainder=2 | hex=5a 00 65 00 42 00 6f 00 6e 00 64 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 03 01 01 00 00 00 00 00 00 00 00 00 00 00 00 32 00 05 33 4c 00 00 00 00 00 00 00 00
start=6238063 | remainder=7 | hex=5a 00 65 00 42 00 6f 00 6e 00 64 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 03 01 01 00 00 00 00 00 00 00 00 00 00 00 00 32 00 05 49 31 00 00 00 01 00 00 00 69
start=6261619 | remainder=3 | hex=5a 00 65 00 42 00 6f 00 6e 00 64 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 03 01 01 00 00 00 00 00 00 00 00 00 00 00 00 32 00 05 49 31 00 00 00 00 00 00 00 00
start=6308734 | remainder=6 | hex=5a 00 65 00 42 00 6f 00 6e 00 64 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 03 01 01 00 00 00 00 00 00 00 00 00 00 00 00 0a 00 05 54 25 00 00 00 00 00 00 00 02
start=6462996 | remainder=4 | hex=5a 00 65 00 42 00 6f 00 6e 00 64 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 03 01 01 00 00 00 00 00 00 00 00 00 00 00 00 0a 00 05 7c d0 00 00 00 00 00 00 00 02
start=6747962 | remainder=2 | hex=5a 00 65 00 42 00 6f 00 6e 00 64 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 03 01 01 00 00 00 00 00 00 00 00 00 00 00 00 32 00 05 a7 6d 00 00 00 00 00 00 00 00
start=6934669 | remainder=5 | hex=5a 00 65 00 42 00 6f 00 6e 00 64 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 03 01 01 00 00 00 00 00 00 00 00 00 00 00 00 32 00 05 c5 fe 00 00 00 00 00 00 00 00
start=6981786 | remainder=2 | hex=5a 00 65 00 42 00 6f 00 6e 00 64 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 03 01 01 00 00 00 00 00 00 00 00 00 00 00 00 64 00 05 c5 fe 00 00 00 01 00 00 00 09
start=7170564 | remainder=4 | hex=5a 00 65 00 42 00 6f 00 6e 00 64 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 03 01 01 00 00 00 00 00 00 00 00 00 00 00 00 0a 00 05 f1 ba 00 00 00 00 00 00 00 02
start=7194120 | remainder=0 | hex=5a 00 65 00 42 00 6f 00 6e 00 64 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 03 01 01 00 00 00 00 00 00 00 00 00 00 00 00 96 00 05 f1 c8 00 00 00 01 00 00 00 20
75 found
The 60 bytes dumped above line up with the tokens here. Definitely not claiming to have the "correct" parsing there, but I've checked a few thousand matches with high accuracy in terms of kill, death, and medal counts per player. One gotcha is that AI kills are not recorded (firefight, minigame, etc.).
I also dumped the highlight events to JSON to double check:
@acurtis166 ah, good catch - I had to fiddle a bit with my C# logic there, but this is what it would look like:
namespace ComponentSearchByteAlign
{
internal class Program
{
public static void Main(string[] args)
{
byte[] data = File.ReadAllBytes(@"X:\drive\decompressed_output22.bin");
byte[] pattern = { 0x5A, 0x00, 0x65, 0x00, 0x42, 0x00, 0x6F, 0x00, 0x6E, 0x00, 0x64 };
List<int> matchPositions = FindPattern(data, pattern);
if (matchPositions.Count > 0)
{
Console.WriteLine("Pattern found at bit positions:");
foreach (int position in matchPositions)
{
Console.WriteLine(position);
}
}
else
{
Console.WriteLine("Pattern not found.");
}
}
public static List<int> FindPattern(byte[] data, byte[] pattern)
{
List<int> matchPositions = [];
int dataBitLength = data.Length * 8;
int patternBitLength = pattern.Length * 8;
for (int bitPos = 0; bitPos <= dataBitLength - patternBitLength; bitPos++)
{
if (IsBitMatch(data, pattern, bitPos))
{
matchPositions.Add(bitPos);
}
}
return matchPositions;
}
public static bool IsBitMatch(byte[] data, byte[] pattern, int bitOffset)
{
// Calculates the number of whole bytes to skip.
// We divide bitOffset by 8 because there are 8 bits per byte.
int byteOffset = bitOffset / 8;
// Calculates how far into the byte (number of bits) we need to start.
// It's the remainder when bitOffset is divided by 8, giving the bit position within the byte.
int bitShift = bitOffset % 8;
// On the above, a good example to visualize the behavior:
// If bitOffset = 10, byteOffset = 1 (skip 1 full byte) and bitShift = 2 (start at the 3rd bit in the second byte - we skip 2).
// We now iterate through every byte in the pattern that is given to
// us when the function is called.
for (int i = 0; i < pattern.Length; i++)
{
// Get the data byte that alligns with the current
// pattern byte and shifts the bits to the left by the
// calculated bit shift value earlier.
byte dataByte = (byte)(data[byteOffset + i] << bitShift);
// If bitShift > 0, include bits from the next byte. This is
// important for scenarios where, for example, we're shifting
// by 3 bits, meaning that part of the data will come from the
// next byte.
if (byteOffset + i + 1 < data.Length && bitShift > 0)
{
// Shifts the next byte to the right by the delta between 8
// and the calcualted bit shift value, aligning it with the
// remaining part of the data byte.
// Note: bitwise OR (|=) is used to combine the shifted parts
// so that we can perform a full byte comparison.
dataByte |= (byte)(data[byteOffset + i + 1] >> (8 - bitShift));
}
// Compare dataByte with the current byte in the pattern
if (dataByte != pattern[i])
{
// Not matching at position. No point in
// continuing.
return false;
}
}
// All bits match
return true;
}
}
}
The results match what you shared earlier. I was cross-checking earlier to see whether any of the components were Bond-encoded, but it doesn't seem to be the case from the last film chunk. The gamertag/XUID combos are also found in other chunks, and notice that the chunk counts are generally aligned with the separators on the timeline (might be something there).
Either way, this is going to be one good blog post on tracking down the data and parsing it out, really appreciate you working through this with me, @acurtis166!
@acurtis166 although in other chunks, if I look by the gamertag, I mostly see two instances at about the same bit position, so I am not sure how that data properly aligns to any of the events in the chunk. It likely is something related to positioning or movement within a given segment.
@dend One thing I noticed in the replication data chunks was that the first byte preceding the first occurrence of each player's XUID appears to increment for the players in sequence:
start | xuid | preceding byte | xuid hex |
---|---|---|---|
4019134 | 2535445291321133 | 00 | 2d c7 ef 5b f9 01 09 00 |
4042784 | 2535412760927018 | 01 | 2a 5b f9 c8 f1 01 09 00 |
4055076 | 2533274860220018 | 02 | 72 6e 29 04 00 00 09 00 |
4067336 | 2535420852629077 | 03 | 55 ee 46 ab f3 01 09 00 |
4079548 | 2535437773107238 | 04 | 26 f0 d0 9b f7 01 09 00 |
4103134 | 2535455288437266 | 05 | 12 aa cf af fb 01 09 00 |
4126338 | 2533274855849476 | 06 | 04 be e6 03 00 00 09 00 |
4150052 | 2535468858644382 | 07 | 9e 97 a8 d8 fe 01 09 00 |
Which made me wonder if players might be referenced by this index elsewhere?
@acurtis166 let me take a closer look at the whole "packet" for player events. I do wonder if there is some kind deterministic header that we can use to properly identify either event references elsewhere or maybe some additional markers.
So, some additional observations, and I am purely looking at the hex editor for this (all byte-aligned):
Here is a binary template for 010 Editor to quickly scan the extracted binary content (assuming the file represents each extracted chunk):
struct HEADER
{
char bytes[12];
};
struct GAMERTAG
{
char bytes[32];
};
struct TYPE
{
char bytes[1];
};
struct TIMESTAMP
{
char bytes[4];
};
struct BUFF_PADDING
{
char bytes[15];
};
struct PADDING
{
char bytes[3];
};
struct MEDAL_MARKER
{
char bytes[1];
};
local int offset = 0;
HEADER header <bgcolor=0x659157>;
offset += sizeof(HEADER);
FSeek(offset);
GAMERTAG gt <bgcolor=cGreen>;
offset += sizeof(GAMERTAG);
FSeek(offset);
BUFF_PADDING bp <bgcolor=cBlue>;
offset += sizeof(BUFF_PADDING);
FSeek(offset);
TYPE type <bgcolor=cYellow>;
offset += sizeof(TYPE);
FSeek(offset);
TIMESTAMP ts <bgcolor=cRed>;
offset += sizeof(TIMESTAMP);
FSeek(offset);
PADDING padding <bgcolor=cBlue>;
offset += sizeof(PADDING);
FSeek(offset);
MEDAL_MARKER mm <bgcolor=0xF7AF9D>;
offset += sizeof(MEDAL_MARKER);
FSeek(offset);
PADDING padding <bgcolor=cBlue>;
offset += sizeof(PADDING);
FSeek(offset);
MEDAL_MARKER mtype <bgcolor=0xFFC0CB>;
offset += sizeof(MEDAL_MARKER);
FSeek(offset);
@acurtis166 for the medal list, were those extracted manually or are you referring to an API-based index? That is - the indices don't match the medal metadata (https://gamecms-hacs.svc.halowaypoint.com/hi/Waypoint/file/medals/metadata.json
) but they are accurate.
I made a quick test site with @acurtis166 SPNKr.py and Dash app. It's very barebones but I am updating it rapidly: https://aapokaapostats.site/stats/ https://github.com/aapokaapo/HaloDashApp
@dend My first thought was to use the medal metadata, as well. When that didn't pan out I started recording them manually. This was mostly by stepping through theater mode. Going to town on bots in custom games yielded a decent amount, haha. To fill in gaps I crawled through match history and compared my missing medal value counts to the corresponding match stats responses.
The mapping from that work is here. I'm still missing some of the harder-to-orchestrate ones.
Looks like a good start @aapokaapo ! The team kill count time series is perfect for visualizing a team slayer match.
@acurtis166 - I might be able to help there. I collected a pretty large corpus of my own matches with OpenSpartan Workshop and might just do an aggregation over them, and then map the medal IDs from the film files to individual medal IDs from the match. Once I have that, I can also open a PR and contribute to your list.
@acurtis166 have you encountered matches where the events for a player are not captured in the last chunk at all?
BTW, I am pulling the XUIDs/GT combos from filmChunk0
instead of parsing events from the last chunk - it contains the pairings in fairly standard formats.
At its core, the core would look like this:
public static Dictionary<long, string> ProcessFilmBootstrapData(byte[] data, byte[] pattern)
{
List<int> patternPositions = FindPatternPositions(data, pattern);
Dictionary<long, string> players = [];
foreach (int patternPosition in patternPositions)
{
int xuidStartPosition = patternPosition - 8 * 8;
byte[] xuid = ExtractBitsFromPosition(data, xuidStartPosition, 8 * 8);
var convertedXuid = ConvertBytesToInt64(xuid);
// We make sure that XUIDs are not some weird values.
if (convertedXuid > 0)
{
int prePatternPosition = xuidStartPosition - 22 * 8;
var bytePrefixValidated = AreAllBytesZero(data, prePatternPosition, 22 * 8);
if (bytePrefixValidated)
{
byte[] gamertagData = RemoveZeroBytes(ExtractBitsFromPosition(data, prePatternPosition - 32 * 8, 32 * 8));
players.TryAdd(ConvertBytesToInt64(xuid), ConvertBytesToText(gamertagData));
}
}
}
return players;
}
The format is like this:
GAMERTAG | Padding | XUID (8 bytes) | Marker 1 | Marker 2 |
---|---|---|---|---|
[GAMERTAG - Dynamic Length] | 0x00 0x00 0x00 ... (22x) | [XUID] | 0x2D | 0xC0 |
@dend I have only run into cases (other than missing AI kill events) where a couple kill/death events might be mysteriously missing for a match. Here's one match: 4841da72-c55a-411f-82fe-31bde5d11b97. I could imagine a case where this could happen for a player that only had one or two events and they got dropped, maybe? It would be nice to load historical matches into theater mode.
That's a good call on the header file. My unverified worry with that was that it would only include players that started the match, but it looks like it would simplify things. One thing I ran into with the highlight events was that after certain game updates the distance between gamertag and XUID positions changed. Also - watch "Marker 1". I do not know why, but rarely I found that marker value show up as 0x25, though the header and event files may differ.
@acurtis166 I'll give you an example match, can you help me validate whether you see the same behavior?
Match ID: 0421e4e3-0fbc-4df5-a3d3-3b4615f2f1e3
You can search for my own gamertag (ZeBond - 5A 00 65 00 42 00 6F 00 6E 00 64
). I don't see anything in the final chunk.
And yes, I have not checked yet whether the players in the film initialization chunk represent the players that were present at the end of the game, although I hope that the film is fully processed when the match is over. I'll see if I can validate that hypothesis with some of the more recent matches with quitters and joiners 😀
@dend I actually don't think there are supposed to be any events. I'm looking at the match stats response and do not see any kills, deaths, medals, or mode-specific counts for your XUID.
@dend Check this out. Near the end of each replication data chunk there is a 4-byte little-endian integer encoded at the position highlighted below:
If you subtract the equivalent value from the previous chunk, you get values that are suspiciously similar to the duration of the chunk from the film metadata JSON. I've seen similar values at the beginning of and throughout the files as well.
index | timestamp? | difference from last | metadata duration (ms) |
---|---|---|---|
01 | 2294366207 | NA | 19993 |
02 | 2314371049 | 20004842 | 20004 |
03 | 2334377860 | 20006811 | 20006 |
04 | 2354387014 | 20009154 | 20008 |
05 | 2374390076 | 20003062 | 20002 |
06 | 2394403283 | 20013207 | 20012 |
07 | 2414408528 | 20005245 | 20005 |
08 | 2434411157 | 20002629 | 20002 |
09 | 2454414409 | 20003252 | 20003 |
10 | 2474417759 | 20003350 | 20003 |
11 | 2494421731 | 20003972 | 20003 |
12 | 2514425010 | 20003279 | 20003 |
13 | 2534437543 | 20012533 | 20012 |
14 | 2554442309 | 20004766 | 20004 |
15 | 2574445700 | 20003391 | 20003 |
16 | 2594449862 | 20004162 | 20004 |
17 | 2614455029 | 20005167 | 20004 |
18 | 2634458286 | 20003257 | 20002 |
19 | 2654464494 | 20006208 | 20006 |
20 | 2674468757 | 20004263 | 20004 |
21 | 2694472161 | 20003404 | 20003 |
22 | 2714478405 | 20006244 | 20006 |
23 | 2729097740 | 14619335 | 14619 |
@dend Dug a little deeper into my last comment. I haven't figured out a consistent method of extracting all the timestamps across matches, but I was able to do so for a single match using a marker that showed up after each timestamp and decoding the prior 4 bytes to see if the value was within an acceptable range relative to the previously decoded timestamp.
Couple findings:
@dend Big team battle:
@dend I actually don't think there are supposed to be any events. I'm looking at the match stats response and do not see any kills, deaths, medals, or mode-specific counts for your XUID.
@acurtis166 - aha, that was one match where I actually left mid-game (I believe it was on a bad map), so that makes sense. But here is another one for you:
Match ID: a60ca05a-c965-439a-b728-46cbdf17a256
Look up this player: 2533274830598241 - BOSS IS TOO PRN
(the hex version is 42 00 4F 00 53 00 53 00 20 00 49 00 53 00 20 00 54 00 4F 00 4F 00 20 00 50 00 52 00 4E
). Do you see anything?
@dend I do see the 60 events. It looks like maybe the the gamertag you are using might be off. I'm seeing "BOSS IS TOO PRO".
Aha! Good catch @acurtis166 - looks like my initial GT extraction logic is flawed when it comes to extraction. Adjusting as we speak.
@acurtis166 found the flaw in my logic - needed to update this piece in the extraction:
extractedData[byteCount - 1] &= (byte)(0xFF >> (7 - endBitShift));
So, confirmed - what's in filmChunk0
are only players that were there at the start of the match. It does not contain players that were there at the end of the match. End-of-the-match players are captured in the final chunk (type 3
).
Here is a fun match to analyze: a9a19635-ab78-446c-8b0a-582949c318cc
.
Here are the XUIDs and players that we've extracted from the film bootstrap chunk:
XUID | XUID hex | GT | GT hex |
---|---|---|---|
2533274855333605 | E5 DE DE 03 00 00 09 00 |
ZeBond | 5A 00 65 00 42 00 6F 00 6E 00 64 |
2535444125582900 | 34 06 74 16 F9 01 09 00 |
Brightbuddy | 42 00 72 00 69 00 67 00 68 00 74 00 62 00 75 00 64 00 64 00 79 |
2535420913081024 | C0 5A E1 AE F3 01 09 00 |
D4RK90HAZE | 44 00 34 00 52 00 4B 00 39 00 30 00 48 00 41 00 5A 00 45 |
2535453467537243 | 5B EF 46 43 FB 01 09 00 |
Kendrothma | 4B 00 65 00 6E 00 64 00 72 00 6F 00 74 00 68 00 6D 00 61 |
2535420184563877 | A5 10 75 83 F3 01 09 00 |
Killa2urDome | 4B 00 69 00 6C 00 6C 00 61 00 32 00 75 00 72 00 44 00 6F 00 6D 00 65 |
2535443540376873 | 29 7D 92 F3 F8 01 09 00 |
SSJ ANGELITO24 | 53 00 53 00 4A 00 20 00 41 00 4E 00 47 00 45 00 4C 00 49 00 54 00 4F 00 32 00 34 |
2535414687920931 | 23 F3 D4 3B F2 01 09 00 |
DrewStylez | 44 00 72 00 65 00 77 00 53 00 74 00 79 00 6C 00 65 00 7A |
2535453506777021 | BD AF 9D 45 FB 01 09 00 |
psilocybe007 | 70 00 73 00 69 00 6C 00 6F 00 63 00 79 00 62 00 65 00 30 00 30 00 37 |
Now, this data does not represent the players that are actually captured by the match stats endpoint:
https://halostats.svc.halowaypoint.com/hi/matches/a9a19635-ab78-446c-8b0a-582949c318cc/stats
Which returns a few more XUIDs that we haven't seen above (the table is complete, including bid
bots):
XUID | XUID hex | GT | GT hex |
---|---|---|---|
2533274855333605 | E5 DE DE 03 00 00 09 00 |
ZeBond | 5A 00 65 00 42 00 6F 00 6E 00 64 |
2535444125582900 | 34 06 74 16 F9 01 09 00 |
Brightbuddy | 42 00 72 00 69 00 67 00 68 00 74 00 62 00 75 00 64 00 64 00 79 |
2535420913081024 | C0 5A E1 AE F3 01 09 00 |
D4RK90HAZE | 44 00 34 00 52 00 4B 00 39 00 30 00 48 00 41 00 5A 00 45 |
2535453467537243 | 5B EF 46 43 FB 01 09 00 |
Kendrothma | 4B 00 65 00 6E 00 64 00 72 00 6F 00 74 00 68 00 6D 00 61 |
2535420184563877 | A5 10 75 83 F3 01 09 00 |
Killa2urDome | 4B 00 69 00 6C 00 6C 00 61 00 32 00 75 00 72 00 44 00 6F 00 6D 00 65 |
2535443540376873 | 29 7D 92 F3 F8 01 09 00 |
SSJ ANGELITO24 | 53 00 53 00 4A 00 20 00 41 00 4E 00 47 00 45 00 4C 00 49 00 54 00 4F 00 32 00 34 |
2535414687920931 | 23 F3 D4 3B F2 01 09 00 |
DrewStylez | 44 00 72 00 65 00 77 00 53 00 74 00 79 00 6C 00 65 00 7A |
2535453506777021 | BD AF 9D 45 FB 01 09 00 |
psilocybe007 | 70 00 73 00 69 00 6C 00 6F 00 63 00 79 00 62 00 65 00 30 00 30 00 37 |
bid(41.0) | N/A | N/A | N/A |
bid(57.0) | N/A | N/A | N/A |
bid(0.0) | N/A | N/A | N/A |
2535409714771833 | 79 B7 68 13 F1 01 09 00 |
DarthChemo31 | 44 00 61 00 72 00 74 00 68 00 43 00 68 00 65 00 6D 00 6F 00 33 00 31 |
2533274943130338 | E2 8A 1A 09 00 00 09 00 |
RestrictedOreo | 52 00 65 00 73 00 74 00 72 00 69 00 63 00 74 00 65 00 64 00 4F 00 72 00 65 00 6F |
2535464010682428 | 3C 90 B2 B7 FD 01 09 00 |
gamerbb3311 | 67 00 61 00 6D 00 65 00 72 00 62 00 62 00 33 00 33 00 31 00 31 |
So there are 3 players that were added to the game at the end. This makes it super-convenient to use the "bootstrap" chunk to learn about who was there at the start of the game and then the end chunk to see who was there at the end. Now, you could arguably also do this by looking at the match endpoint metadata and correlate between PresentAtBeginning
, LeftInProgress
, and PresentAtCompletion
, but films make it easy to also correlate those with events.
Now, the next important step was identifying how to reliably extract Gamertag + XUID combos from the "final" chunk, given that the structure is slightly different than in the "bootstrap" chunk.
The pattern for the XUID identification is the same (I am running with 0x2D 0x2C
for now, assuming that 0x25
is not frequent). Looking at byte-aligned data, the gap between GT and XUID was different from piece to piece and might represent different things. For example, the first instance of 0x2D 0xC0
had 2535453467537243
(5B EF 46 43 FB 01 09 00
) XUID next to it, and the GT before it was psilocybe007
, but that's clearly not the right XUID for it (should be 2535453506777021
, i.e., BD AF 9D 45 FB 01 09 00
). Within the same "chunk" of data, there is also a reference to 29 7D 92 F3 F8 01 09 00
, which is 2535443540376873
(SSJ ANGELITO24
) - also not the right association. So looks like from the final file we can't really quickly map out GT+XUID from different message chunks, since they are not always associated with each other. However, that same metadata format that I explained earlier - GT+XUID combo in /filmChunk0
is also available in other chunks (not the last one).
Getting Halo Infinite Match Stats With Official Halo API · Den Delimarsky
Getting visibility into your match performance outside the game.
https://den.dev/blog/halo-api-match-stats/