Error unpacking SSD (Metal Gear Survive)

[Joey35233]

Attempting to unpack any .dat from SSD yields this:

Unhandled Exception: System.IO.EndOfStreamException: Unable to read beyond the end of the stream. at System.IO.BinaryReader.FillBuffer(Int32 numBytes) at System.IO.BinaryReader.ReadUInt32() at GzsTool.Core.Qar.QarEntry.Read(BinaryReader reader) at GzsTool.Core.Qar.QarFile.Read(Stream input) at GzsTool.Program.ReadArchive[T](String path) at GzsTool.Program.Main(String[] args)

I throws this error on line 92 of QarEntry, on QarEntry() #2.

The files can be found here.

[TinManTex]

The smallest dat (300mb) if you just want to poke at that: https://mega.nz/#!fZliWS4a!zWzFu_r8-xbRlFmPTSHEcxCrLjo99tGepnTganQKVCE

[unknown321]

Breaks on chunk0.dat after the patch, zlib error:

D:\users\unknown>J:\mgs\gzssurvive\GzsTool.exe "G:\SteamLibraryWin2\steamapps\common\METAL GEAR SURVIVE BETA\master\chunk0.dat"
45ada1c810e4.dat
197cd03f88edd.fpk

Unhandled Exception: Ionic.Zlib.ZlibException: Bad state (unknown compression method (0x34))
   at Ionic.Zlib.InflateManager.Inflate(FlushType flush)
   at Ionic.Zlib.ZlibCodec.Inflate(FlushType flush)
   at Ionic.Zlib.ZlibBaseStream.Read(Byte[] buffer, Int32 offset, Int32 count)
   at Ionic.Zlib.ZlibStream.Read(Byte[] buffer, Int32 offset, Int32 count)
   at Ionic.Zlib.ZlibBaseStream.UncompressBuffer(Byte[] compressed, Stream decompressor)
   at Ionic.Zlib.ZlibStream.UncompressBuffer(Byte[] compressed)
   at GzsTool.Core.Utility.Compression.Uncompress(Byte[] buffer)
   at GzsTool.Core.Qar.QarEntry.ReadData(Stream input)
   at GzsTool.Core.Qar.QarEntry.<>c__DisplayClass43_0.<ReadDataLazy>b__0()
   at GzsTool.Core.Common.FileSystemDirectory.WriteFile(String filePath, Func`1 fileContentStream)
   at GzsTool.Program.ReadArchive[T](String path)
   at GzsTool.Program.Main(String[] args)

data1.dat unpacks without a trouble, contents are full of garbage.

[TinManTex]

Nice start Atvaark, thanks. As said above, data1.dat extracts, but file contents are still encrypted.

[Atvaark]

There's still another layer of XORing that's done to the files before the compression layer. I found the function that I think does the XOR (sub_140062460) but I still have to understand that assembly and port it to c#.

[Atvaark]

[LatGeek]

Very nice! I'm curious to poke at those textures. Some of the new visual design looks great. Looking forward to the release.

[TrikzMe]

Here is what I've got when I try to unpack "chunk0.dat" But ftextool is working and scripts files are well decrypted ;)

[emoose]

Excellent work! I tried having a stab at it but couldn't get very far at all, thanks for working it out so quickly!

I've started adding SSD filenames to MGSV dictionary project too, seems most of this game is reused from TPP so it's not too difficult to find the names (really hope the final game will be more than just a TPP clone though)

(btw, does anyone know what SSD stands for?)

EDIT: seems some files don't fully decrypt, see 24e187dc382eb.lua / 29ae3908e36f3.lua inside data1.dat for example

[BobDoleOwndU]

(btw, does anyone know what SSD stands for?)

My best guess was (Metal Gear) Solid Survive Demo. I don't know if that's accurate though.

[Atvaark]

I need the file extension name that match these hashes: 1531 7263 (wma, wmv, mp4 or something video related I guess)

[emoose]

1531 = gkn / hhy (both give 1531) 7263 = wmv

[Joey35233]

Slightly straying off-topic here, but @emoose, you might want to check out TinManTex's mgsv-lookup-strings.

[unknown321]

Speaking of strings, I have two memory dumps - online and main menu. Strings from them can be found here - https://mega.nz/#F!h0wzlRJQ!R5iMLhULQF6HhLocGWrAdQ

[Joey35233]

data1.dat makes the game not respond when repacked and it appears there are some character encoding issues (for example with foxfs.dat) and purportedly with some luas.

[Atvaark]

releases/v0.6.0-beta

[Joey35233]

So what can't this do yet?

[Atvaark]

So far I only checked if I could unpack the dat and convert the ftex texture files to dds.

• There are no Survive exclusive file names
• Booting the game with any repacked files isn't tested. Better unpack and repack with empty dictionaries in case Konami changed the file name hash algorithm
• There appear to be some more custom encrypted files (like the lua files) e.g. the new "gkn" files

[Joey35233]

Ah. When you say that there are no Survive exclusive file names do you mean that the dictionary isn't updated?

[emoose]

All seems to unpack fine now, thanks for the fix!

Also looks like the 1531 "gkn" extension is actually "param", as seen in TppDefine.lua: "/Assets/ssd/level_asset/server/parameter/SsdWeaponParameters_opt.param"

EDIT: also seems the param files get updated from a Konami server, guess that's why they're encrypted. Hooking the cityhash functions shows it hashing the URL for them followed by another string being hashed that looks like the key, no idea if that key would match up with the .params in the QAR or if it's just for the web params though: https://pastebin.com/MPfzSGmq

[Atvaark]

Likely their custom blowfish with a hardcoded key again. (Like https://github.com/unknown321/mgsv_emulator)

[Joey35233]

Hey, @Atvaark, @emoose, do you want to come to our Discord? We have all of the researchers there and I think that it would be easier to discuss issues there.

[emoose]

Ah thanks for the info @Atvaark, you were right - what I thought was a key was actually just an md5 of the param file, the key was hardcoded in the exe as you guessed.

For anyone interested I've uploaded the decrypted param files along with the scripts I used to decrypt them (Blowfish algo sourced from https://github.com/unknown321/mgsv_emulator). Follow the link at https://pastebin.com/mqT1fCr8 (edit: updated with beautified JSON + missing encoder_consts.py I forgot to include...)

In a nutshell the files are zlib compressed and then encrypted with that custom blowfish algo, pretty simple. Not sure if you might want to add support in GzsTool to automatically decrypt the ones inside the QAR, would be a nice addition, but maybe not much point since they'll just get replaced with the server-side ones anyway (and SSD is always online ;_;)

[BobDoleOwndU]

@emoose Could you explain how you got the param files from Konami's server? (Especially ones that aren't included in Survive's files like SsdSbmParameters.)

I'm trying to get the latest version of them; but I'm not sure where to begin outside of capturing traffic from the game.

Edit: Found the address. It's http://mgssd-game.cs.konami.net/ssdstmweb/parameter/

[rokoni99]

Hi friends,

I have problem with GzsTools When I try to unpack pes files (fpk) Gives Error message

Like this

non-negative number required. parameter name value.

Please help..........