Document the WoW file formats

[ghost]

mangos-zero heavily relies on several data files used in the World of Warcraft 1.12.x client.

We need to create clean format documentation in order to ensure that we use correct data when creating custom data files based on the WoW client files. An example is map, vmap, and mmap generation.

Archive and compression

• .mpq: archive format. The format's capabilities include compression, encryption, file segmentation, extensible file metadata, cryptographic signature and the ability to store multiple versions of the same file for internationalization and platform-specific differences. MPQ archives can use a variety of compression algorithms which may also be combined.

  Database

• .dbc: client database.
• dnc.db: Day-Night cycle specification.
• .lit: lightning information.
• .sbt: cinematic subtitles.
• .trs: minimap hash translation.
• .wdb: client data cache.

  Graphics

Graphics are mostly 3D models that allow building models in real-time or non real-time 3D rendering.

Raster graphics

• .blp: texture format, supporting texture compression, and multiple resolutions.

  3D graphics

• .bls: instructions for pixel and vertex shaders.
• .m2: Blizzard Entertainment's own model format, also known as '''.mdx'''.
• .wfx: assigns shaders to surfaces.
• .wmo: world map object file, containing either a group definition or a single world map object.

  Geographic information

• .adt: terrain and object information for map tiles.
• .wdl: low-resolution height maps.
• .wdt: map and object placement information.
• .zmp: map tile reference.

[ghost]

Please note that documentation should be added to the server repository wiki on github.

[shlainn]

Can't access the wiki - do I have to be a member of this thing?

[ghost]

I just enabled the Wiki.

[shlainn]

Status Report:

Low hanging fruits are picked. ADT, WMO, BLP and M2 will follow shortly after careful rechecking. I implemented readers for the latter three in PseuWoW ages ago - what a pain.

[ghost]

Finally had the time to read through the first documents and thus here comes feedback:

• DBC files could be extended by describing the supported column types.
• MPQ files might benefit from links to stormlib, libmpq (both on github), and smpq on launchpad.
• DNC.db = I like :)
• SBT and TRS files, the same.
• WDT description might benefit from a more formal approach, probably tabular view of the structures.

[shlainn]

• DBC: Actually, I was thinking to describe the column types for each DBC file specifically. I stated in the Table Data section, that most values are 32bin unsigned integers, which is true for 95% of the values. Rarely there is a float or an uint16
• MPQ: Feel free to add them ;)
• WDT: please specify _formal approach_. The problem with a tabular view for structures (I assume you are referring to something like http://en.wikipedia.org/wiki/IPv4_header#Header) is that for the very repetitive file chunks this will not help. While we are at it: who is the target audience for this?

I have just finished ADT Files, please take a look. We should hammer out the formal approach you would like to see before I dive into M2, BLP, WMO or any other painful file format

[ghost]

I have to dig into my archives, I think I wrote something for MPQ already a while ago.

[shlainn]

bump

[ghost]

What I had in mind was something a little more like e.g. the PNG format description on Wikipedia. I like the structure of the technical details a lot.

[shlainn]

Added some polish to https://github.com/mangos-zero/server/wiki/WDT-files and https://github.com/mangos-zero/server/wiki/Chunked-file-structure - I may be too stupid to understand what you are trying to tell me by that Wikipedia Article... ;)

[shlainn]

BLP is in, although I am not incredi-happy with it. It's mostly copypasted from Wikipedia. I did not recheck it in code, although considering that the BLP reader in PseuWoW is based on this and is working with BLPs from 1.12.x I consider this valid.

[shlainn]

Started on M2 files. It would be nice if anyone could read the stuff I wrote (also the other articles) and tell me if there are parts that are unclear.