feat: support for halo infinite

[Surasia]

• [x] Add support for resource indexing
• [x] Properly show index of resources
• [x] Add meta plugins for Meta Viewer
• [x] Bitmap support
  • [x] Bitmaps with embedded data
  • [x] Bitmaps that include sequences
  • [x] Tags with indexed resources
• [x] Model support (thanks gravemind!)
  • [x] RTGO support
  • [x] Proper mesh weights
  • ~Particle models~
• [x] Loading known tag names from a text file
• [x] HD1 (high-res textures pack) support
• [x] Merge H5 and Infinite module loader
• [x] Scenario support
  • [x] Loading runtime geo
  • [x] Material loading
  • [x] Removing shadow casters
• [x] Support for all infinite versions
• [x] Coating support
• [x] OCGD (Object Customization) support
• [x] Unify stringmapper/tagmapper

[Surasia]

look into: modules are not being read properly. A proper implementation (seems) to be here, however I cannot get it to work with Reclaimer.

@Connor-jt I'm not sure if you're still interested in Halo stuff, but could you expand on how you read module header and module files? Currently, my implementation looks like this, and I also tried this, which worked, however tags did not get decompressed properly.

I assume it is related to the header alignment, though even after adding that, it does not seem to function.

[Connor-jt]

theres a good chance you might have made a minor mistake in the data you're reading if every tag fails to decompress then i would guess that yeah you aren't aligned correctly with where the tag data actually starts at?

easiest way to figure it out would be running through your code that doesn't work alongside some code that does work. like just opening up the same module in the two different tools and matching up notable offsets like the tag data offset thing.

the tag reading is annoyingly complicated at least, so its almost expected to make a few mistakes when putting together a reader for it

ill get back to you after looking through the code here

[Connor-jt]

this doesn't look right i dont see any other issues though, although i didn't double check your tag structures to make sure they're all correct

um yeah for the tag reading/indexing process itself, i mean its pretty straight forward once you have the right tag structures and something to cast raw bytes into those structures

since reading module files is really as simple as reading the bytes into the

• module header
• module item structs
• the string table (which isn't included anymore?)
• resources indicies
• data blocks headers and then you calculate the amount of padding that separates this header stuff from the tag data region

and then i believe the process for reading at least the headers for individual tags was something like

• lookup module item's data blocks (data block index & data block count), if no data blocks owned by tag then just fetch the data from the values the module item header gives you (just the data offset & compressed/uncompressed size)
• then access the data from data blocks from the module tag data offset + module item data offset + compressed offset
• decompress that data if its marked as compressed
• assemble all data block's data into single array?
• cast array to tag header if its a tag file and not something else like a manifest file or resource file or whatever

i didn't look through the code for that process though, so its probably a bit more detailed than that, and i cant quite remember it all right now

[Surasia]

@Connor-jt Thanks a lot! The 8 byte padding was indeed the issue. Everything else (including resource indexing!) works now 👍

[Surasia]

Git can be a pain sometimes... now properly pushed the changes.

[Surasia]

Bitmaps can now mostly load. Textures that have bitmap data contained inside of them seem to be a bit tricky.

Also, some files use the HD1 modules from the HD texture pack. These are found as stubs with empty data inside the original module, with a flag to indicate if the additional modules should be used. Currently, I will simply use one less mip to load the proper data- adding the HD1 module support in the future.

[Surasia]

Self-contained bitmaps now fully work.

[Surasia]

Added the option to provide a file path to a .txt file that contains file names. It should be formatted like this: (on each line) 00002B36 : fx\library_olympus\objects\characters\marines\_parts\marine_handcuffs_lightning_ring_beam.particle

This isn't ideal because reclaimer uses Little Endian integer global IDs, while the tag path uses a big endian hexadecimal string to indicate global tag ID. In the future, I want to avoid having to parse an entire file (something compressed and built into reclaimer as a resource)

Currently, it simply goes through each line and maps it to a "TagMapping" dictionary with a key of int and value of string, which at least makes it rather fast to load.

[Surasia]

Around 90% of bitmaps now load. The majority that don't load are R32G32B32A32 bitmaps (which I don't plan on adding) and some bc7 sequenced bitmaps.

[Surasia]

oops, accidentally pushed ci stuff to this branch. removed it now

[Surasia]

converted back to draft, got some more things i want to add

[Surasia]

@Gravemind2401 I think this is pretty much everything that I think would be best fit for an initial release. I wasn't really able to think of a way to properly implement coatings, so I think I will leave it for now. The biggest issue I have currently with this PR is the string/tag mapper; it does not have any functional issues however it requires a very specific format of a text file that is not documented anywhere, or given an example of. I might remove it for now, but I'd appreciate if you could weigh-in on this.

[Gravemind2401]

@Gravemind2401 I think this is pretty much everything that I think would be best fit for an initial release. I wasn't really able to think of a way to properly implement coatings, so I think I will leave it for now. The biggest issue I have currently with this PR is the string/tag mapper; it does not have any functional issues however it requires a very specific format of a text file that is not documented anywhere, or given an example of. I might remove it for now, but I'd appreciate if you could weigh-in on this.

I'm not really sure what the best way to handle the strings would be. You could create a dedicated repository for the strings file so at least there's a common place people can get it from and you can push updates to, but in terms of implementation I guess a text file is as good as any.

As for merging, there's still a couple things I want to sort out when I get around to it. Currently the tag hierarchy still uses the four-character class codes which is different to the existing behaviour for earlier games. Since there's no strings I'll have to just generate a tag class list xml like what Halo 1 and 2 are using. I'd also like to implement a proper module interface similar to ICacheFile so that the tag viewer and meta viewer functionality doesn't need to be duplicated.