Koei Tecmo .bin/.idx Archive

[DarkSynopsis]

Issue: Plugin Request Console: 3DS Format extension(s): .bin/.idx Type: Archive Game Name: Hyrule Warriors / Fire Emblem Warriors

First 8 byte of file(s): bin - [26 00 00 00 34 01 00 00] - [&...4...] idx - [4C E2 04 01 00 00 00 00] - [L.......] Taken from FE: Warriors, different for each.

More details: Koei Tecmo games appear to use .bin/.idx files, there is a QuickBMS script (http://aluigi.altervista.org/bms/hyrule_warriors_legends.bms) that will extract the files but doesn't keep file names or folder structure and I've no clue if repacking works, had no luck with that side of QuickBMS in the past.

Sample files (if possible): Fire Emblem Warriors (From Update) https://mega.nz/#!Ih4gwZDA!zoNgP36E2_ITf_0FyuIt1Mdl9oXaIVW5w6m9xhhbUMw

Hyrule Warriors (From Update) https://mega.nz/#!ZtJAkApR!zCYWSHeBQfbGV_eGpJpIEysaqiJfEzfkvDrk-WkqUFY

Used updates for smaller file sizes to test with.

[kuronosuFear]

<Duplicate commen from #530 >

Hi! Bumping this request.

Sangokushi might look the same as some other files in #346 but they don't use compression, so this certainly needs its own Kuriimu\Karameru implementation

I've made my own file extractor (no re-compiler function... yet), but of course having it compatible with Kuriimu is a different beast. https://github.com/kuronosuFear/Sangokushi-Xtractor/releases

Offset 0x0000 - Marker [1D 78 33 01] Offset 0x0004 - Size of Table [DD 4A 00 00] (0x00004add); Note needs to be multiplied by 0x10 for the correct table size. Offset 0x0008 - Alignment [20 00 00 00] (0x00000020); To get the correct offsets/address per entry in the table, use this as a multiplier Offset 0x000c - Null Offset 0x0010 - Table's first entry

Table Structure (16 bytes per entry)

[entryAddress : dword][null : dword][rawSizeOfData : dword][decompressedSize : dword]

Size of the Table = 0x00004add0 (306,640) Number of entries = 19,165

There are two kinds of data: Compressed (uses zlib: best compression [78 DA] header) and Uncompressed/RAW

The uncompressed/raw files can easily be extracted, and it can be identified by checking if the decompressedSize in the entry is 0.

Meanwhile, the compressed data has its own custom structure:

[decompressedSize : dword][firstChunkSize : dword][firstChunkData:chunkSize] [secondChunkSize : dword][secondChunkData : secondChunkSize] . . [lastChunkSize : dword][lastChunkData : lastChunkSize] All starting offsets of the entries are aligned to 0x20, so KoeiTcmo does some padding if it's compressed data falls a little bit short.

Each zlib compressed chunk corresponds to 0x8000 bytes of the actual file. (It must be a resource/memory saving technique for the 3DS)

Writing a re-compiler seems very feasible.

So far, the only thing Karameru/Kukkii supports are the extracted *.sarc files.

Me and @NeonToaster1 identified where the text data are, but it isn't compatible with Kuriimu at the moment. (The extracted files do not have a unique marker on their own. We're guessing that they're referenced by a different file somewhere)

Hopefully these data/findings help Sangokushi in being supported by Kuriimu.

[onepiecefreak3]

As per my research, Yo-Kai Sangoukushi does not have a .bin/.idx file pair included in them. I only found the LINKDATA_A.BIN, which is a X3 archive, referenced in #530 . This is misleading, so I remove the mention of this game from the main issue text.

[onepiecefreak3]

A plugin for this format was added to the newest dev build and upcomgin Release 1.2.1 of Kuriimu2. It's a manual plugin, which means that the plugin has to be select manually, since the files cannot be detected automatically. Please select the plugin "BIN_IDX" under "plugin_koei_tecmo.dll" to open this file format. Issue closed.