HaoJun0823 / GECV

5 stars 0 forks source link

i need help on how blz2 extraction works #11

Closed DarkGD123 closed 6 months ago

DarkGD123 commented 6 months ago

Just finished reading @nachotacos69 WikiEater related to GE2 1.40 DLC (DECRYPTED

i tried copying the blz2 part on the dlc even though there is alot Screenshot (148)

then pasted them in a new file then i had this issue when i use the tool

> GECV_EX_BLZ4.exe unpack "blz2_1.blz2" "blz2_1.blz2.new" blz2
GECV EX BLZ BY RANDERION(HAOJUN0823)
https://blog.haojun0823.xyz/
https://github.com/HaoJun0823/GECV
Unhandled exception. ComponentAce.Compression.Libs.zlib.ZStreamException: inflating: unknown compression method
   at ComponentAce.Compression.Libs.zlib.ZOutputStream.Write(Byte[] b1, Int32 off, Int32 len)
   at GECV_EX.Utils.BLZ4Utils.CopyStream(Stream input, Stream output) in F:\Documents\VS2022\GECV\GECV II\GECV_EX\Utils\BLZ4Utils.cs:line 105
   at GECV_EX.Utils.BLZ4Utils.DecompressData(Byte[] inData, Byte[]& outData) in F:\Documents\VS2022\GECV\GECV II\GECV_EX\Utils\BLZ4Utils.cs:line 91
   at GECV_EX.Utils.BLZ2Utils.BLZ2FileDecompresser.GetByteResult() in F:\Documents\VS2022\GECV\GECV II\GECV_EX\Utils\BLZ2Utils.cs:line 237
   at GECV_EX.Utils.BLZ2Utils.UnpackBLZ2Data(Byte[] blz2_file) in F:\Documents\VS2022\GECV\GECV II\GECV_EX\Utils\BLZ2Utils.cs:line 49
   at GECV_EX_BLZ4.Program.Unpack2(String blz, String file) in F:\Documents\VS2022\GECV\GECV_EX_BLZ4\Program.cs:line 116
   at GECV_EX_BLZ4.Program.Main(String[] args) in F:\Documents\VS2022\GECV\GECV_EX_BLZ4\Program.cs:line 32
nachotacos69 commented 6 months ago

im not sure thats how it works, like the copy pasting part 🤔

nachotacos69 commented 6 months ago

PATCH.indx.txt

i made a little list where to find blz2 only read the START part.. and avoid checking the END, since i made it for other purposes

HaoJun0823 commented 6 months ago

I just verified this, uploaded the new version, and I've managed it based on feedback from others. BLZ2 uses a different compression stream than BLZ4. Since I don't have the game files, I only tested whether it functions normally and did not try to test the game files.

At the moment I don't know if the files are all working properly, because BLZ2 doesn't have strict data validation, and I can only guess at how it works.

nachotacos69 commented 6 months ago

@HaoJun0823 I'll do the testing, I'll try to report after I fix some internet issues lol

HaoJun0823 commented 6 months ago

PATCH.indx.txt

i made a little list where to find blz2 only read the START part.. and avoid checking the END, since i made it for other purposes

Both BLZ4 and BLZ2 are compressed through a series of zlib methods.

BLZ4 decompresses/compresses the data stream through ZLIB, while BLZ2 uses DEFLATED.

The header of BLZ4 is four-byte BLZ4: The decompressed size of four bytes (actually eight bytes, but the game does not have a single file larger than Int32, and this part of the design is also used in the file sections of PS4 and PSV). Eight bytes of 0, (see above, it should be four bytes of 0) Sixteen-byte MD5, So the header file has a fixed size of 32 bytes.

The BLZ2 header is four bytes of BLZ2 with no additional verification data.

Next, it consists of multiple blocks, each block has a 2-byte block size (UINT16) and block data. Read the block size in sequence, and then read the block data according to the block size until the end of the file.

Since BLZ uses 16-byte alignment in some places (such as RDP), it should end when reading a block size of 00.

Then, the game uses a strange order to store these chunks, for example the order of 0,1,2,3,4, in BLZ the order is 1,2,3,4,0, so you need to reverse it and pass it through zlib in order Stream decompression/compression.

For BLZ2, when the file is less than or equal to 0XFF, no compression is performed, so there is only one block. In the file data set of the PRES file, if the file is compressed by blz, a decompressed file size will be recorded for allocation in memory to solve the problem that blz2 does not record the decompressed file size. For BLZ4, some files are not compressed, but these files may have multiple uncompressed chunks.

Some of the original data blocks will become larger through the zlib compressed stream, so my blz4 strategy sets each block size to 32768, while blz2 keeps the size of 0xff online.

HaoJun0823 commented 6 months ago

I think my V4 version completely solves the problem of compressing/decompressing BLZ2 files. BLZ2's data blocks have special compression format settings that I can't adjust through Microsoft's libraries so I can only decompress them. So I didn't find a library in .NET to process files. I wrote a python version based on the py file that someone gave me before.

nachotacos69 commented 6 months ago

Using GOD EATER 2 1.40 DLC, i use my self made tool just to just rip some blz2 stuff inside the PATCH.EDAT file

Before Screenshot (152)

After Screenshot (153)

DarkGD123 commented 6 months ago

Closing this one..

Yamato should send something like a request later on @HaoJun0823