VGAGraph support is mostly hard coded

[Blzut3]

It looks like most of the loading is hard coded based on the number of chunks. This isn't exactly needed as the order is more or less known so it's possible to determine the type by examining the information given. There are a few catches though.

• The first chunk is the pictable. The length of this chunk is hard coded so some editors fill in the length with garbage. However, since the size of the compressed data is known (chunkOfs[1] - chunkOfs[0]) we can rely on the huffman decoder to give us a pointer to the end of the data. Derive the size from that to determine the number of graphics in the VGAGraph.
  • As an additional sanity check, scan the pictable and truncate if an image is reported as 0x0 or larger than some arbitrary size. (I use 640x480 as the max size in ECWolf.)
• Following the pictable will be some number of fonts. Unfortunately the only way to know how many fonts is to go one at a time and check to see if the chunk can be interpreted as a valid font. (ECWolf does this check in the ECBlake branch.)
• After all the image chunks is the TILE8. This chunk has no uncompressed length since vanilla just assumed it contained 72 tiles. This assumption is bad since S3DNA has 106 tiles (IIRC). For this, pick a max size (I use 256 tiles) and then use the method from before to run the huffman decoder on the compressed buffer. The end pointer will tell you the size which can be divided by 8*8 to get the number of tiles.
  • With my knowledge limited to Wolf3D engine games, I do not believe there are any VGAGraphs which contain a masked TILE8, TILE16s, or TILE32s. Allowing these to exist would make things a little more difficult, but from what I can tell they would have to be in that order so you could attempt this method on the chunks following and see if the output size makes sense.
• Following the TILE8 can be either palettes or text screens (ENDOOM). It looks like they can come in either order, but the size of these would be constants.
• After these is either demos or ASCII text chunks. I believe the demos are always grouped together, but they could be before, after, or in the middle of the text chunks.

[Gaerzi]

Yeah, that was based on various #defined constants in the Wolf3D source. If I had written that code later, I'd probably have used the crc instead of the number of lumps to identify them.

But why couldn't id have used the VSWAP format for everything, really? Or something like CMP... Supporting the archives from other "id engine" than the Wolf3D-clones would be a bit too much out of scope to be worth the effort, since they tend to like compressing everything and squirreling the head and dictionary in the game's executable, and then packing the executable for good measure. (I did look into what would be needed to open Keen's audio.ck* files, and then promptly chose not to bother.)

[Blzut3]

Well they could have used VSWAP for everything, but they wanted to just reuse the tools they already wrote for Commander Keen. Of course the VSWAP file was created to handle memory pages which could be swapped in and out of EMS/XMS. (Hence the name.)

I do agree that supporting every game that used the file formats is a bit out of scope for Slade, but as far as reading the VGAGraph goes, the method outlined in the report should allow proper detection of contents without any hard coded values (just ignore the TILE16/32 point). Doing so would have the advantage of allowing Slade to open any "vanilla" Wolf data as opposed to just supporting the ones you become aware of. (In that sense I would consider using CRC worse than what you currently have since it would only work for the exact data file.)

Outside of the last two steps, you can observe ECBlake's loader for reference. Or I guess wait awhile and eventually I'll submit a patch to fix the loader.

While slightly off topic, since you're bringing up audio here, I will mention that S3DNA uses a 11,111Hz sample rate for its digital sounds in the VSWAP. Not sure if you care to support that oddity, but I guess the least intrusive way to support it would be to have a cvar to change the rate. I can't think of a way to distinguish between S3DNA files and the others.

[Blzut3]

I forgot, Blake Stone has an additional compression type (LZH) on a few chunks in the vgagraph. Unfortunately the first of such chunk (POWERBALL) appears after the TILE8, but before the error screens/palettes and then a few more follow the screens/palettes. Which means they need to be considered part of that group. The good news is, these can be identified easily by a fourCC of JAMP (for JAMPacked) after huffman expanding.

I also poked at the other commercial data files a bit. It turns out Corridor 7 and Operation Body Count do not have a TILE8. So before assuming a TILE8 one must check where the length is supposed to be reported and see if it's 4008, 4007, or 5 (more on this in a second). If so it should be safe to assume these are 80x25 screens. While on the topic of the TILE8, when decompressing to try to find the size, you'll get a number that isn't a multiple of 64 usually. I just truncate the garbage bytes (&(~63)) in ECWolf so that should work fine.

As for the fixed sizes for the screens. The commercial data uses 4000, 4008, or 4007 bytes. The source code for Wolf3D seems to suggest the first 7 bytes are skipped, but I might be wrong there. Palettes are always 768 bytes.

Now as for Operation Body Count. It has neither screens nor palettes. Instead it has two 5 byte garbage chunks at the end. I believe it's safe to just ignore them, especially since Slade only intends to few the files. Only the floppy version has the chunks encoded properly according to the vgadict, the CD version has the exact same data for the chunks even though the vgadict changed (so uncompressing them fails). When determining the presence of a TILE8 I would check for a reported size of 5 and that the offset difference (compressed size) is 6 (10 if you count the 4 bytes for the size).

[Gaerzi]

This is going to be a mess of heuristics and special cases. Not that the WolfArchive code wasn't already a mess.

[sirjuddington]

I'm assuming this isn't going to be done any time soon? Should probably move it off the 3.1.1 milestone if that's the case

[sirjuddington]

Closing - old issue

Feel free to reopen the issue if needed