cemu-project / Cemu

Cemu - Wii U emulator
https://cemu.info
Mozilla Public License 2.0
7.11k stars 554 forks source link

[Feature Request] Lossless conversion tool+ #568

Open SamRohod opened 1 year ago

SamRohod commented 1 year ago

A standalone tool (or within Cemu) that can do the following conversions (preferably in bulk) losslessly:

.wud/wux <----> .wua

Extracted Folder <----> .wud/wux

This way we can store all the Wii U content as .wua and be able to convert them back to their original dumping format (for hash verification). Making the .wua format truly archival (everything can be turned back to the way it was originally dumped).

I'm just an idiot. The rest of this is just me guessing without actual expertise

If I understand correctly, this might need the user to input a key for .wud/wux --> .wua but if the key can also be stored in the .wua file, then .wua --> .wud/wux can be possible without a key.

I don't know if there are random seeds for the .wud file padding or not, but tools like WudCompress can convert between .wud and .wux with archival quality (hash doesn't change for the .wud file). So, I assume the padding has a pattern to it.

This might also need some understanding of the files layout of the original .wud/.wux file. Something like IRD files for PS3 discs (layout information). Cemu is already capable of reading .wud/.wux files, so this might be possible. If this layout information can be stored in the .wua file during conversion and also be exported to a standalone file (like IRD files for the PS3), then maybe the following conversions could be possible for disc games:

.wud/wux + key file/text + Layout file ----> .wua ----> Extracted Folder (disc game)+ key file/text + Layout file Extracted Folder (disc game) + key file/text + Layout file ----> .wua ----> .wud/wux + key file/text + Layout file

In regards to updates and DLCs in Extracted Folder form, I don't know if they can be converted back to CDN format. If so, an implementation within Cemu for the following conversions would be great:

Extracted Folder (Update/DLC) + ??? ----> CDN format ----> .wua .wua ----> CDN format ----> Extracted Folder (Update/DLC) + ???

With all the previously mentioned conversions and the ability of .wua format to hold multiple contents, the .wua can actually be used for archival/preservation proposes of a full game with all of its updates and DLCs in one file while remaining playable on the emulator. Which seems like a good general standard for all "retro" preservation. Maybe also add the original hashes of all the files in the .wua file in the file itself for quick verification without conversion.

Squall-Leonhart commented 1 year ago

WUA doesn't retain information required to unpack the contents to their original state.

SamRohod commented 1 year ago

WUA doesn't retain information required to unpack the contents to their original state.

Is the conversions/processes I mentioned possible with a different or new format/modified .wua?

Bitwolfies commented 1 year ago

Ah, so that's why there's no way to convert back, it not being lossless is a huge bummer. Is there any chance at least to be able to recreate the original files using something like redump, simmaler to nkit? Mostly worried about future mods that cannot use the current cemu graphic patch system, and I really don't wanna redump everything. (EX: TMS decensor patch)

Exzap commented 1 year ago

WUA is lossless. In the same way that zip, tar, etc. is lossless. In general you can think of it like .zip but with the added ability of random-access. It's just a compressed container for a collection of files.

The loss of information is when you convert from WUD/WUX/.app (aka FST) to extracted files. The conversion drops the meta data and cryptographic signatures not relevant to game execution.

So in a way the problem is independent of WUA. If someone made an extractor tool that keeps around all the meta data and the filesystem tree from the original FST input then it would also work with WUA, because like .zip it's just file storage and whatever you input into it you losslessly get back. The trickiest part is making a tool that can reassemble all the loose parts back to the original FST format. But again thats independent of WUA.