WAR audio format from Win32 demo

[ostrosablin]

I've pulled and extracted an audio.spk from [DEMO] 7 Days. Salvation.7z archive. It seems to contain a bunch of *.war files.

Not sure if this format is used in any other known versions of the game, but as far as this version is concerned - WAR files appear to be 16 KHz 4-bit Little Endian VOX ADPCM encoding.

https://en.wikipedia.org/wiki/Dialogic_ADPCM

Here's an example of e_train_far.war sound, decoded and re-encoded into a more common audio format.

e_train_far.zip

[minexew]

Nice find, we can add this info to the README. Did you use ffmpeg for the conversion, or custom tooling?

It would be interesting to see if the game executable calls Windows APIs for audio decoding, or if they embedded the codec.

[ostrosablin]

Actually, I've imported them into Audacity as raw audio. I've used following settings:

Encoding: VOX ADPCM Byte order: Little Endian Channels: Mono Offset: 0 Sample rate: 16000 Hz

My build of ffmpeg seems to reject the files, even though codec (adpcm_ima_oki) should be supported. Perhaps, wrong commandline.

Unfortunately, this version seems to lack music, found in most other builds, only SFX seems to be present (e_train_far is longest one).

If you could point me, how to extract data archives from *.app Dingoo A320 build, I would take a look at SAU format, mentioned in README. But I bet it's also some variation of ADPCM encoding, because they're simple to decode and don't use a lot of CPU time.

[ostrosablin]

In meantime I've dealt with bulk conversion of war sounds to generic wav. This seems to be doable with SoX in a single bash line:

for i in *.war; do sox -t ima -r 16000 $i -t wav $i.wav; done

I suppose "war" stands for "raw" in reverse, which could be an indication of little-endian encoding.

[minexew]

If you could point me, how to extract data archives from *.app Dingoo A320 build

Honestly, I don't know. A320 uses the same CCDL executable format implemented in https://github.com/minexew/Gemei-RE/blob/master/Tools/readccdl.py, but I'm not so sure if they also embed resources in the same way (ERPT section).

[Yoti]

If you could point me, how to extract data archives from *.app Dingoo A320 build

HEX editor ;) From the 0x00150000 to the EOF without last byte (0xFF).

[ostrosablin]

Regarding SAU format (extracted from res.spk of 7days_v1_176208.rar):

First of all, it requires chi variant of extraction algorithm (and also, this archive actually contains nested paths, so it also required path-handling fix to the SPK extractor).

python3 ./extractspk.py res.spk ./test4 --variant=chi

Also, my hunch was right, it's also a variant of 4-bit ADPCM encoding. Moreover, it seems to be very close to WAR files, and even could be decoded to WAV with same SoX command. Though, something seems to be not right, as decoder produces warnings:

sox WARN adpcms: t_swordbreak.sau: ADPCM state errors: 145

Converted files sound pretty good to me, but looking at resulting WAVs in Audacity, there seems to be some clipping and amplitude is significantly higher, than in Win32 build. I'm not sure, if this is defect of original audio (could it be Dingoo Games crudely amplifying the original signal for this version?) or if they have somehow modified decoding algorithm.

[Yoti]

according to Dingoo SDK. The description file, the file name is CCDL and there are several types of headers:
CCDL, IMPT, EXPT, RAWD, [ERPT], [<null>]

Each Header is 32 Bytes long and has the following format:
char     name[4]; // 4-byte ASCII chunk name
uint32_t ident;   // Chunk identifer
uint32_t offset;  // Offset of this chunks data area in the file.
uint32_t size;    // Size of this chunks data area in the file (usually padded to 16-bytes).

43 43 44 4C|00 00 01 00 01 00 02 00|04 00 00 00 CCDL = 4
20 08 12 17 19 23 16 00 00 00 00 00 00 00 00 00
49 4D 50 54|08 00 00 00 A0 00 00 00|88 08 00 00 IMPT = 888 => 890
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
45 58 50 54|09 00 00 00 30 09 00 00|40 00 00 00 EXPT = 40 => 8D0
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
52 41 57 44|01 00 00 00|70 09 00 00|E0 9C 13 00 RAWD / Type 1 / @970 / =139CE0
00 00 00 00 20 6A AD 80 00 00 A0 80|70 42 14 00
00 00 00 00|00 00 00 00 00 00 00 00 00 00 00 00 NULL
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00

[Yoti]

Regarding SAU format

SAU player with src in Delphi: https://www.cyberforum.ru/post14721034.html

[ostrosablin]

@Yoti wow, awesome!

I've re-implemented the decoding routine from SAU player in Python and made it a fully-functional standalone script to convert SAU files into regular uncompressed PCM WAV files. Uploaded it into Tools/decodesau.py.

After all, my initial assumption about AMI ADPCM scheme was wrong. While it sounds about right, but in sound editor, it can be clearly seen that the SAU files, decoded by SoX look wrong (as opposed to ones, produced by decoder from delphi SAU player).

It's most noticeable if you try to loop the files, produced by SoX (as I've tried) - at the loop point there would be a loud audible click, because signal drifts off-center (while there's no such artifact, produced by SAU player).

[Yoti]

I've re-implemented the decoding routine from SAU player in Python and made it a fully-functional standalone script to convert SAU files into regular uncompressed PCM WAV files.

May you do something like this but for .war? It's same as .sau but with another byte endianness (0x12345678 <-> 0x21436587) and without zero byte at EOF.

[ostrosablin]

May you do something like this but for .war? It's same as .sau but with another byte endianness (0x12345678 <-> 0x21436587) and without zero byte at EOF.

Yes, no problem. It would be just a matter of adding an option for byteswapping the input data.

However, I wouldn't be so sure about last zero byte. It seems to be true for some SAU files. But not for all. Here's example:

xxd m_tension_01.sau | tail -n2
0000ecc0: 8a00 2125 4312 0810 3247 818a d9aa a9cc  ..!%C...2G......
0000ecd0: 2936 1200 8f                             )6...

As we can see, in _m_tension01.sau last byte is 8f. And this isn't the only example. There's actually a lot of these. So we can't really go and strip last bytes from SAU files. I guess it really depends on the input. Here's an example of 7Days_Piano01.sau:

xxd 7Days_Piano01.sau | tail -n2
00050b50: 0000 0008 0080 0080 0000 0000 0000 0800  ................
00050b60: 1980 0080 0000 0000 0008 0000 0800       ..............

We can really see the 00 byte at the end, but if you look carefully - there's actually lots of zero bytes around the EOF. So I would actually assume that it's just statistically (depending on actual waveform) sometimes happens to be on the EOF. Hence, I didn't strip last byte from SAU files in decodesau script, because I'd speculate that it's also a sound data.

I might be wrong, though, but it looks that way.

[ostrosablin]

I've updated the decodesau.py script. Now it decodes both WAR and SAU. e.g.

Plain SAU:

python3 ./decodesau.py ./test4/audio/e_train_far.sau 2.wav
Reading input file...
Decoding SAU input...
Writing the output (165512 PCM frames)...
Finished!

Nibbleswapped WAR:

python3 ./decodesau.py ./test/e_train_far.war 3.wav --byteswap
Reading input file...
Byteswapping the input (WAR -> SAU)...
Decoding SAU input...
Writing the output (165512 PCM frames)...
Finished!

Comparing the files (WAR and SAU, where both variants are available) again seems to show, that they're byte-to-byte identical, except that their bytes have nibbles in swapped order (there's no explicit zero byte added at EOF), so I guess it's a wrong assumption, after all.

[minexew]

Excellent work. Maybe we can make it even more user-friendly by guessing the nibble order from the file extension (and require explicit selection if the extension is not recognized)?

[ostrosablin]

@minexew Yes, that would be logical. Implemented in 258f98f

New behavior is to auto-infer nibble order from extension, and if it cannot be detected - to explicitly specify it via --byteswap or --no-byteswap.

[ostrosablin]

Some tech thoughts:

It turns out to be IMA ADPCM, after all (I've compared decoder code). And actually, it's SAU that seems to be nibble swapped, not the WAR. WAR's nibbles are actually 12, and SAU is 21. SAU decoder goes from lowest to highest nibble (21), not other way around (12). This also means that we could've get away with implementing just a nibble-swapper, and decode to wav with SoX, but anyway, having a standalone and user-friendly converter is a good thing.

Actually, this looks like a kind of optimization for sake of performance, because 21 order might look unnatural and "reversed" to human reader, but for machine, it would be very natural to read, because you can essentially mask off high bits (byte & 15) to separate first sample. And then shift the original byte right by 4 positions - and that would be a ready-to-decode second sample. That's just two cheap operations.

In fact, WAR's nibble order would take more operations and/or memory to read samples in "12" order. Because you either need to nibble-swap it beforehand (that alone is 4 operations), or separate it by copying and shifting high bits, and masking high bits off in an original byte. That would be 3 operations. So, naturally, for performance-reasons, there's clear advantage to byteswap resources themselves, not the data in runtime. I bet this was reason for swapping bytes in SAU format.

This also means that we can pretty easily produce both WAR audio and SAU audio (e.g. to replace sounds/music resources with custom sounds). It's just a matter of encoding 16khz ima-adpcm mono sounds with SoX. For WAR, ima-adpcm sounds could be used as is. To encode SAU files, SoX output also has to be nibble-swapped, but that's about it and should work.

[ostrosablin]

I've uploaded a small program nibbleswap.py to the Tools. This is, essentially, a reduced variant of decodesau.py, which has decoder removed and only nibbleswap functionality kept. This (together with SoX) should allow to easily convert sound resources for 7 Days: Salvation to allow replacing original sounds.

Producing WAR sound example (just plain IMA ADPCM, really) from any format, supported by SoX:

sox modem2.wav -t ima -c 1 -r 16000 modem2.war

Now, to produce SAU sound from encoded WAR - you need one more step:

python3 ./nibbleswap.py ./modem2.war ./modem2.sau

That should swap nibbles and make file readable by the newer builds of the game.

The reverse is also true - if you want to convert SAU to WAR, for instance, you could invoke nibbleswap.py on SAU file, and it would produce a WAR/IMA file, which just works with SoX and other programs that understand VOX/IMA/Dialogic ADPCM encoding, as well as being native to older builds of the game.

I've attached proof-of-concept encoded sound resources, obtained with above method. Didn't test them in game, but they should work just perfectly, because we now have a good understanding of what both WAR and SAU is.

encoding_example.zip