[Help]: .unit format discovery thread

[Xaymar]

Everything about the .unit file structure.

Known Information

• X is Up, forward is either Z or Y
• Vertex Data:
  • Types:
    • 0: Position
    • 1: Color or Normal
    • 4: Texture Coordinates
    • 6: Unknown
    • 7: Unknown
  • Formats:
    • 2: float32[3]
    • 26: uint8[4] or sint[4]
    • 29: float16[2]
    • 24: Unknown
      • Only seen together with type 6 or 7. Usually no space left at this point.

Related links:

• https://reshax.com/topic/507-helldivers-2-model-extraction-help/

[Xaymar]

This mesh should have animation data: 0x990B45D5D75FFF3A, 28. File contains errors #4, #6 and #3. The values in 0x7, 0x1f suggest it is indeed (type_2b_wide)[4], but the order is not exactly described. It looks like it'd be half, though it is unclear what the 4th value is for. Most of the time, it is 0, but rarely it is non-zero in the "exact" amount that's missing from [nX, nY, nZ].

[Xaymar]

"Up" in the Mesh is X, "Forward" is unknown.

[Xaymar]

0xA1AD36AF446FCB7D, mesh 6: Position is offset by 8 bytes, but there's no noticable difference to other structures. In this file, offset 0x18C into the DataType structure is set to 1. The vertex stride here is still 1, so what is the difference?

It's 100% identical it terms of elements:

The only noticable difference is the single bit set to 1 suddenly:

[Xaymar]

The offset is exactly 8, and only necessary for the first vertex. After that, all data is consecutive again.

[Xaymar]

This difference is not present in the other variant of the same model, which does not have this offset. Not related to the datatype?

[Xaymar]

This dword is different at the end.

[Xaymar]

That same dword is different in every model, so probably not that?

[Xaymar]

Oh, there appears to be a lot more data for meshes that is currently ignored. Not quite sure how to parse it at all...

[Xaymar]

This may be a possible BMS export error, and not actually caused by my tool.

[Xaymar]

We've had the mesh structure all wrong, and didn't notice it until now.

1. The "offsets" entry is not relative to count, but relative to the list.
2. Each mesh starts with two uint32s.
3. There can be multiple materials.
   • 0x64 is the number of materials, 0x68 is the offset into the structure -4.
4. There can be multiple vertex groups.
   • 0x74 is the number of groups, 0x78 is the offset into the structure.

This is big. It means that several meshes were incomplete by default, since we assumed there was only one set of vertex/index information. I'm not sure why the vertex_offset and vertex_count repeat as well, but they haven't changed yet.

[Xaymar]

0xA1AD36AF446FCB7D = 0x7DCB6F44AF36ADA1. Original BMS script had this set to BigEndian, but it's all LittleEndian. There are three copies of this file throughout the various data files, and only one of them has the 8 extra bytes at the start. The other two are fine.

Is there something in the data file that tells us more about why this is happening maybe?

[Xaymar]

Original BMS script is correct, it's easier to work with when it's big endian since the file name matches the bytes in the file. It's more like the id is 8 bytes long, so when we read it as an uint64_t it just ends up flipped.

[Xaymar]

Hashes appear to be MurmurHash(2) 64bit version. Not quite sure how to implement it, but I'll focus on proper exports for now.

[Xaymar]

When Type 6 or 7 is encountered, the data does not fit into the vertex stride anymore. They cannot reasonably exist in the same space as everything else.

[Xaymar]

The matrices and vectors in __unk1 seem to be related to something. There's some kind of relationship tree happening there, but I don't quite understand how it works yet.

[Xaymar]

Similarly, __unk6 seems to belong to the same type of relationship.

[Xaymar]

The end of unk1 overlaps with unk5 if __unk5 is present.

[Xaymar]

No apparent relation between bones file and __unk1. Number of bones doesn't ever match up with the number stored in there.

[Xaymar]

I'm probably approaching this from the wrong angle. I'm assuming that the game engine uses a model format, but this is clearly multiple models in one. It could very well have merged all the meshes into one cluster. It doesn't quite explain where animation and bone data goes though.

As far as I can tell, each of these .unit files is usually accompanied by .bones, .state_machine, .physics and .dfde6a8797b4c0ea. For units without a .bones file, the ones that are present are .physics and .dfde6a8797b4c0ea. For units with a .bones files, the .dfde6a8797b4c0ea is replaced with .state_machine often.

[Xaymar]

We also see .336bdb8766bd591d accompanying the same file - possibly another part of the .unit file.

[Xaymar]

Also of note is that several of the files are very clearly Havok Physics, and they all seem to be related to the bones in the model.

[Xaymar]

May have found some sort of LoD or similar?

unk0.unk_data[].__unk_data[].data[] is always lower than the maximum amount of meshes. Seems to cluster together around meshes.

[Xaymar]

__unk5 appears to also limit itself to the number of meshes in the unit file.

[Xaymar]

Unfortunately my theory about type 7 and 6 not fitting in the vertex stride was destroyed by another file having a vertex stride where they fit again.

[Xaymar]

Format 24 might be "Bone Indices" and Format 25 is "Bone Weights". These both appear to be u8vec4_t

[Xaymar]

Format 24, 25 and 26 are still unknown. 25 is definitely not bone indices, as either there are 65535 possible bones, or most models refer to bone 255 and assign a weight of 0 to it - assuming 24 is bone weights.

[Xaymar]

Formats:

• 1: float[2]
• 2: float[3]
• 4: ???
  • Can't be longer than 4 bytes.
• 24: ???
  • Can't be longer than 4 bytes.
• 25: ???
  • Can't be longer than 4 bytes.
• 26: ???
  • Can't be longer than 4 bytes.
• 29: half[2]

Types:

• 0: Position
  • Known formats: 2
• 1: ?
  • Known formats: 26
• 4: Texture Coordinates
  • Known formats: 1, 29
• 5: ?
  • Known formats: 4
• 6: ?
  • Known formats: 24
  • Related to .bones file
• 7: ?
  • Known formats: 25
  • Related to .bones file

[Xaymar]

content/objectives/obj_common/gneeric_intel/generic_intel.unit

• No .bones file, and entry for .bones is empty.
  • Coincidentally, no mention of type 6 and 7 suggesting a relation between the two.
• Type 1 is still present.
• Type 5 is also present.

[Xaymar]

content/fac_helldivers/vehicles/lav/lav.unit

• Proof that gpu_resources is treated separately: All data is off by 4 bytes with 8 byte offset, but it's off by -4 in other files if we use 12 byte offset.

[Xaymar]

content/fac_helldives/vehicles/combat_walker/combat_walker.unit

• __unk1.__unkf32mat4x4_1 may be Translation, Rotation and Scale in one Matrix?
• __unk1.__unk_u16vec2_0 may be u16vec2_t determining some sort of relationship between nodes.
  1. Unknown, seems to always be 0 or 1?
  2. The parent node to attach to.
• __unk1.__unkcommonhash seems to be the hash table for nodes.
  • This is suggested by the 7th mesh having a hash entry that is at location 102 in the above table.

[Xaymar]

We now have a reasonably deep understanding to reproduce a mesh tree in glTF format.