jackz314 / bars-to-bwav

Extracts BWAV files from BARS files (usually used as sound containers in Nintendo Switch games)
MIT License
13 stars 0 forks source link

The contents of some .bwav files from some .bars files are not matching with the .bwav file names after extracting them #4

Open K-E-R-A-D opened 4 years ago

K-E-R-A-D commented 4 years ago

Is this because of the way the code written or the contents within some .bars file seem to be, unorganized? here's an example of a file itself: $Melody_Npc_Key07_Steps.zip

the contents of the .bwav file itself is actually an eating sound in general, and the .bwav file itself was extracted straight from a file Npc.bars from the game animal crossing new horizons, with your tool. when opening the .bars file in hex editor program, the file name Melody_Npc_Key07_Steps is found on an offset 1715 to 172A. whereas the file name for eating sound (the actual filename for that .bwav content file is supposedly Npc_Action_Eating_General) found on offset 21F3 to 2200. (i unfortunately unable to locate the .bwav file of eating sound itself for the supposedly corresponding file name)

P.S.: as for the tool, i built the tool myself by copying all the codes you made from bars-to-bwav.cpp, then i built them using Code::Blocks IDE. It has c++17 compiler as well (GNU and ISO). i have trouble using the built executable you made. built, and tested on windows 7

jackz314 commented 4 years ago

The thing is, my way of getting the file names for these bwav files is probably not the best way to do it, and it'll probably have problems outside of the small set of bars files I tested on. Until someone can figure out the actual structure and precise patterns of these bars files, there probably will be problems in the filenames. If you figured out the file structures, please let me know and I can update the code accordingly.

As for the built executables, I'm not sure if it'll work on Windows 7, but I just pushed some changes that should get rid of the need for dev libraries on Windows 10 machines, it's in the release page if you want to check it out.

K-E-R-A-D commented 4 years ago

Hmmm...

after looking at several .bars files, i think the most possible cause is that one .bars file contains more AMTA tags than the number of BWAV files that the .bars file has. for example:

  1. Npc.bars (the .bars file used for (most likely) the sound of villagers interact with something or so) contains over 135 AMTA tags, while there are only 77 BWAV files contained. which 58 of the AMTA tags are a mystery.
  2. Doubutsugo_Aneki.bars (uchi / sisterly villager animalese audio files) contains over 241 AMTA tags and 236 BWAV files, which 5 of them is also a mystery.
  3. Doubutsugo_Bonyari (lazy villager animalese audio files) also contain over 241 AMTA tags and 231 BWAV files, which 10 of the AMTA tags are also a mystery.
  4. or perhaps worse, Pl_Footstep.bars (the player footstep sounds) contains 1680 AMTA tags and 1108 BWAV files, 572 of AMTA tags are a mystery.

i presume more than one AMTA tags use, or link to the same BWAV files, or some of them are unused AMTA tags

any .bars files that has an equal number of amount of AMTA tags and amount of BWAV files contained within do not have this issue

on another note, looks like i finally found where the sound Npc_Action_Eating_General located. it's on offset 81A40 to 8FABF in Npc.bars that is...

also, by the way, is the code working with this sort of, tags? just asking because i'm not sure if it does

OceIdC4RYu (it is found in Keyboard_Boy_en.bars, a keyboard sound played when typing in english keyboard, as a boy)

there are 36 AMTA tags found in Keyboard_Boy_en.bars. (i believe) these are in QWORD format. separate them and you get 36 QWORDs, matching with the number of AMTA tags

K-E-R-A-D commented 3 years ago

sorry for the double comment but i finally found (sort of, but hopefully) a lead

remember the tags i showed you above? those are actually the QWORDs of file offset set(s). following this wiki page: https://github.com/Kinnay/Nintendo-File-Formats/wiki/BARS-File-Format#file-offset-set , i can tell what are those mean (although it may be made for Wii U, but i'm really sure it's also relevant to switch as well) (i also wonder if you are aware of that page above in the first place)

here's an example of the QWORD i meant 68 05 00 00 80 AF 00 00

the first DWORD (68 05 00 00) represents an offset to audio meta file / AMTA tag while the second DWORD (80 AF 00 00) represents an offset to asset file / the BWAV file

and after reassess the QWORDs, i think i found the possible cause: in some .bars files, some of the first DWORDs related to the same second DWORD, which should explain why more than one AMTA tags related to the same BWAV file, which also explains that one (or some more) .bars file contains more AMTA tags than the number of BWAV files that the .bars file has.

an example: 9Kupb7egLv

(yet again, found in Keyboard_Boy_en.bars . the file alone has 36 QWORDs of file offset sets, and 35 BWAV files. the reason i chose Keyboard_Boy_en.bars is an example of a simple case of the cause.) (to make it looked like above, i use a hex editing program (HxD), then i separate them all to a new file. Don't bother with the offsets on the picture above though. The actual starting offset of the file itself is from, like 0xA0

As for the starting offset of file offsets in other .bars files, they're relative, which is not always the same. Another example of this also found in Doubutsugo_Aneki.bars , which has a starting offset of 0x3D4 within the file (but same starting offset also found in Doubutsugo_Bonyari.bars , Doubutsugo_Futsu.bars , Doubutsugo_Genki.bars , Doubutsugo_Hakihaki.bars , Doubutsugo_Kiza.bars , Doubutsugo_Kowai.bars and Doubutsugo_Otona.bars). And as for the end offset of them, it will only end when the hex begins with AMTA tag (41 5D 54 41), like this one found in Doubutsugo_Aneki.bars (so the DWORD of 00 76 38 00 actually counts as offset to asset file / the BWAV file for the DWORD of 68 7A 00 00 , full QWORD code will be 68 7A 00 00 00 76 38 00 )

Also i'm terribly sorry to say this: i'm no programmer, i'm just a user who just trying to point possible things out to help you... and i'm also sorry for messing up any terminology above, like the QWORD, or DWORD, and making too many edits from the previous comment

jackz314 commented 3 years ago

Nice find! The first link you showed works on older Wii games, and some of the earlier Switch games as well, but Nintendo has probably since changed their file structures because the more recent BARS file structure is completely different from the older ones.

The second part about the tags looks very interesting, I took a look at some of the files and they seem to be matching pretty nicely with the offsets there, I haven't figured out a clearer pattern yet, but I'll update once I do.

As for the terminologies, don't worry about them, to me they are not important at all, you've done a terrific job analyzing the situation, especially as a non-programmer.

Update: after looking more into the BARS file's header, it seems that its header structure is the same as the old ones, it's just that everything else is different, especially the metadata (AMTA) section, looks like Nintendo started using a new version of the AMTA, previously it was version 4, now it's version 5, so its structure is different.

K-E-R-A-D commented 3 years ago

after looking more into the BARS file's header, it seems that its header structure is the same as the old ones, it's just that everything else is different, especially the metadata (AMTA) section, looks like Nintendo started using a new version of the AMTA, previously it was version 4, now it's version 5, so its structure is different.

my word, i forgot about that part if that's the case, this might explain how i gotten into this way when i tried to extract the BWAV files from FtrWeddingOrgan.bars (a .bars file for an instrument Wedding Pipe Organ, added in update 1.2.0 of the game, and ofc that file was added right in the update so it won't be found in the base game) DRMrV7WmMU

or this, an attempt to extract BWAV files from Fireworks.bars (pretty self explanatory, a .bars file used to play fireworks sound, added in 1.4.0 update) q0cYIV1rBG

in short, may run into an issue when extracting .bars files that was added in the update. i highly suspect that the version 5 of AMTA was made to, sustain for update needs (advanced ones perhaps?). and somehow, different .bars files added on a different update have different starting offset of file names...

by default, the code indicates that the 0x48 is a starting offset of file name in the AMTA tag HxD64_2HGDlvvY18

But turns out the AMTA tags in .bars files added in the updates are a different story. Here are several examples of AMTA tags w/ different starting file name offset (in a different .bars files that added on a different update)

(this one is found in FtrWeddingOrgan.bars , again, a .bars file for an instrument Wedding Pipe Organ, added in update 1.2.0 of the game) HxD64_tYqFTFKP6R

(this one is found in FtrDiveFishMizukurage.bars, a .bars file that plays um, the jellyfish swimming sound in a display, i mean like, a small aquarium, a furniture of some sort, added in update 1.3.0) (and yes the jellyfish is obtained by swimming / diving in the sea) HxD64_1kXW03DGIa

(this one is found in FtrPirateCannon.bars , a .bars file that plays some sounds when interacting with the pirate cannon furniture, added in update 1.3.0) HxD64_IbuQtyFPDk

notice the starting offset of the file above, they're different compared to the .bars files mainly found in base game. the starting offset of them is 0x56 (while the offset from 0x50 to 0x54 indicates the update version [0x55 seems to be a null that acts like a space button on keyboard, of some sort])

and last, but not least, this is found in the Fireworks.bars (again, pretty self explanatory, a .bars file used to play fireworks sound, added in 1.4.0 update) HxD64_kL2U7woUxk

there, what baffles me is that the starting offset is 0x4C...

quick short note: another actual starting offset of file name is 0x56, and another one is 0x4C (not sure if there might be more, unless, an absolute structure of AMTA tag version 5 has been figured out) but i think that's all for the time being, it might have another different offset of starting file name of an AMTA tag in .bars file once a new update hits

EDIT: i was told by someone some time ago about a kicker of AMTA v5

look at offset 0x24, it has a value that indicates an offset the starting file name, and here's the kicker: perform a hex addition operation, 0x24 + the value on offset 0x24 itself

e.g.: from the AMTA of Fireworks.bars like an example above, the value on offset 0x24 is 28, and therefore 24 + 28 = 4C and therefore, the offset of starting file name is 0x4C

i highly believe this applies to .bars files, found in both base game and updates

K-E-R-A-D commented 3 years ago

double comment again because it's relatively different topic

The second part about the tags looks very interesting, I took a look at some of the files and they seem to be matching pretty nicely with the offsets there, I haven't figured out a clearer pattern yet, but I'll update once I do.

can i help again with that part? i think i finally figured out those code thingy. frankly it took me a while to figuring it out somehow...

just as what this page mentioned, they are an absolute offset of each of the DWORD thing, except there are some, differences

take a look at this picture i uploaded here a few weeks ago: (the offset QWORD code found in Keyboard_Boy_en.bars) 9Kupb7egLv

and take one QWORD code as an example:

68 05 00 00 80 AF 00 00 it means the AMTA tag on an offset of 0x568 is related to the BWAV file on an offset of 0xAF80. i know it's kinda strange for both DWORD codes (68 05 00 00 and 80 AF 00 00) because they read in reverse (right to left, ignore the 00 on the right ones). like in these pictures: HxD64_OxDfUSpgxJ (offset of the AMTA tag, 0x568)

HxD64_lcuD2DlQIB (offset of its related BWAV, 0xAF80)

take another example of the QWORD code: 08 07 00 00 00 F7 00 00 it means the AMTA tag on an offset of 0x708 is related to the BWAV file on an offset of 0xF700. and again, they read in reverse (right to left for both of the DWORD code, ignore the 00s on the right ones, the 00 on the left of F7 counts as part of the offset however). like in these pictures: HxD64_5ClS7NvD9y (offset of the AMTA tag, 0x708)

HxD64_2C5GqygsDJ (offset of its related BWAV, 0xF700)

and yes this applies on all of the .bars files in the game, either from base game, or updates. if you wish to look for the starting offset of arrays of those QWORD codes, simply look at the hash code thing, which is starting from 0x10 as mentioned on that BARS file wiki page. those are in DWORD however, matching with the number of AMTA tags available in .bars file itself. Because the starting offset of arrays of those QWORD codes is completely relative, and won't be the same. For example, in Keyboard_Boy_en.bars HxD64_agfwQQdrA5

as you can see on the picture, the hash code thing will be A9 44 66 1B , B0 80 0B 1C , 31 DA CD 21 , and so on until AE 69 DE FC. there are 36 hash code things, matching with the number of AMTA tags there. (oh and those are AMTA's CRC32 hashes, not BWAV's apparently)

alright i hope this helps even more...