burninrubber0 / YAP

A simple bundle extractor and creator for Burnout Paradise.
3 stars 0 forks source link

Remove known memory types and alignments from output #2

Open burninrubber0 opened 1 week ago

burninrubber0 commented 1 week ago

YAP is currently set up to output resources as follows. The key is the resource ID, the type is a resource type ID, and the alignment refers to the per memory type alignment after being loaded into memory.

resources:
  0xdd03acf5:
    type: 0x0
    secondaryMemoryType: 2
    alignment:
      - 0x4
      - 0x80
  0xe289d7f4:
    type: 0x21
    alignment:
      - 0x10

This system was put in place specifically to ensure that non-retail builds, including ones that aren't in our hands yet, would be supported regardless of any changes that may have been made regarding memory types and resource alignments. However, the downside is that it requires the user to have knowledge of memory types and alignments. Even if this can come from other resource metadata and the wiki, it's not ideal.

The ideal file looks like this instead (with the ID as a value rather than a key, in part to aid in #1):

resources:
  - id: 0xdd03acf5
    type: 0x0
  - id: 0xe289d7f4
    type: 0x21

Based on limited research, I believe this could be achieved while still maintaining support for the aforementioned potential differences by:

  1. Tying the (default) secondary memory type to the bundle platform (and perhaps resource type, though it'd only apply to PS3 shaders),
  2. Tying the (default) alignment(s) to the resource type (on a per-platform basis, as alignment changes between platforms), and
  3. Outputting any differing secondary memory types or alignments to distinct files.

This should result in retail files having the simpler output shown above, while any differences will be retained.

The per-platform, per-resource-type alignments will need dumping using a standalone tool to confirm they're really the same between all resources of the same type. smilingfrog conducted similar research in 2017, so I'm fairly confident this will work out favorably.

burninrubber0 commented 5 days ago

The per-platform, per-resource-type alignments will need dumping using a standalone tool to confirm they're really the same between all resources of the same type. smilingfrog conducted similar research in 2017, so I'm fairly confident this will work out favorably.

This has now been dumped (along with some other things as a sanity check) and the results are mostly as expected. There are some standouts:

The first point warrants identifying ShaderProgramBuffer resources by their shader type rather than their resource type, preferably on all platforms. This is how the bundle debug data identifies them (VertexShader and PixelShader). It'll require reading the first little bit of the resource but that shouldn't be too bad performance-wise. In the same vein, it may be time to list types by name rather than ID in the YAML output.

Remastered's switching of alignments wasn't previously known but doesn't come as a surprise, nor does the fact that the change was half-assed and not applied to all resource types. Remastered on PC, at least, is identical to the original PC game. It should be fairly easy to detect when a bundle is PS4 or NX (or presumably XB1) based on the alignment being 4 or 8.

The last point is weird and out of line with everything else I've seen. It's not even consistent across Remastered versions; PS4 kettle and PC uses 4-byte alignment, while PS4 non-kettle and Switch use 1-byte alignment. (Wonder what XB1 uses?) I can only imagine the latter is a bug and will have to be output as though it isn't the default. The 64-bit platforms otherwise use identical alignments between them.