nesrak1 / UABEA

c# uabe for newer versions of unity
MIT License
1.1k stars 143 forks source link

Contribution (Split files) and question (import audio) #226

Open IlluminatiWave opened 1 year ago

IlluminatiWave commented 1 year ago

Summary. I found how to split the files (.split0, 1, etc.).

Quick: Download this [File Splitter], select the joined file and hit split in 1 Megabyte parts (or depending on the size of the original split, all files but the last one have the same size).

It should look like this

"package name "+"part number "+("total of parts").Extension

For example

My original file: sharedassets16.assets Size: 46.7 MB (49,016,856 bytes) Total parts: 47 (46 of 1024kb and one of 765kb) See the hashes section at the end See the hashes section at the end **

To translate from split mode to unity mode, you only need to change the name.

You delete the part of the parenthesis with total parts and at the end, you add ".splitn-1".

Example: Split: sharedassets16_32(47).assets (part 32). Unity rename: sharedassets16.assets.split31


Explanation: Apparently unity doesn't split the files with strange encryption or weird stuff, it just creates the file and splits it without further ado.At the end unity, when required, puts all the pieces back together again. At the end unity, when required, puts all the pieces back together again.

quick example

Stuff (file)

asdfghjkl

splitted (3)

file 1: asd file 2: fgh file 3: hjk file 4: l

I really don't know why (or maybe it was lucky that my "attacked" developer doesn't use some unity function I don't know about, but that it serves to split with extra encryption, I don't know).

Maybe you can adapt the code of that repository so that the UABEA can interpret when a file is split (for example, when creating the decryption, also create an extra file so that when it is detected, simply split it in that amount of parts. Or add a suffix to the name, so that when UABEA exports it again, it simply splits it into that number of parts.

Example: Stuff[1024KB].asset (split this file into equal parts of 1024kb, I think it goes without saying that this can be ignored for the last file).

Hash Stuff Also, to check that this has really worked. I decided to pass all the files under MD5. All parts matched (split0 with its twin _01(47) for example).

checksums.txt

Come to think of it, to avoid corrupted files, the program could first check if the file is "splittable" by simply temporarily splitting the generated file and compare it with existing files. Nothing really difficult and that the user should not know beyond the program telling him that the file can be reliably split).


Regarding the audioasset.

Neither UABE nor UABEA have support for importing audio files, but this possible implementation seemed curious to me, I don't know if it really works, but it deserves a look.

And in case it works, implement it, don't you think? Maybe as a start it could be a good idea to use the fsb as mentioned (asset ripper allows exporting as fsb).

nesrak1 commented 1 year ago

Split files

I already support combining split files into one file. I haven't modded any games with split files, but I was under the assumption that if you just removed the split files and kept a single large file in the game, that it would still load that correctly. Correct me if I'm wrong here. That's why I don't have any functionality to split again since I haven't felt it was all that important.

Audio import

Asked in #136, #181, and on dev board. The primary issue is two things:

  1. Current fsb library (which isn't using official fmod stuff, which is a plus) doesn't support encoding
  2. Audio must be in a separate file outside of the assets file unlike textures, so I'd need to somehow manage editing a resource file which sounds like... not fun. We already know how to import audio manually, but making it user friendly and allowing multiple imports without making a bunch of individual files in the bundle is the difficult part.
IlluminatiWave commented 1 year ago

I already support combining split files into one file. I haven't modded any games with split files, but I was under the assumption that if you just removed the split files and kept a single large file in the game, that it would still load that correctly. Correct me if I'm wrong here. That's why I don't have any functionality to split again since I haven't felt it was all that important.

Wow, I was expecting some kind of error when merging the files, but it has worked correctly the game. However, I guess someone should find it helpful to keep the files split (Some game that checks the files). So it may not be mandatory, but as an optional tool, it never hurts.

Because it was several files that I merged. There should be an option to merge the split0's of a folder and delete the split variants in batch form (it's a bit tedious to merge 20+ split files).

  • Current fsb library (which isn't using official fmod stuff, which is a plus) doesn't support encoding
  • Audio must be in a separate file outside of the assets file unlike textures, so I'd need to somehow manage editing a resource file which sounds like... not fun. We already know how to import audio manually, but making it user friendly and allowing multiple imports without making a bunch of individual files in the bundle is the difficult part.

The game I am editing saves all the music information in assetbundle (.ab) and sharedassetbundle files.

In case you are interested, it's called cytus 2.

The structure is that each .ab that I want to edit, has internally 3 text files (the levels) and one music file (fsb). I can edit the text files, but not the .fsb (I mean I can't put a new fsb, which internally has an ogg of 44100hz). So I don't know what to do in this case to force a custom ogg (or if it could be done so that the game recognizes these files without encoding).

The game has video clips but fortunately it is not necessary to do anything else because these are exposed as mp4 without more (which allows me to make a patch for fullhd / sd textures).

I think the ideal would be to find a way to make the game work with all files decrypted, but unfortunately the main file (sharedasset0 / the one that manages everything) is inside the apk, which for worse, has an apk integrity verification system (an android anticheat, something annoying). So modding at the moment is limited to video clips, some textures and dialogs,

nesrak1 commented 1 year ago

and one music file (fsb).

Are you saying the bundle literally has an fsb in it or are you just talking about the resource file?

Like I mentioned before, the primary issue is updating the resource file when multiple audio assets are modified. Besides that issue, following the guide that you posted should work. It's just automating it and guaranteeing you can't break something is another issue. I suppose I can make a different bundle entry for every new audio asset to avoid dealing with rearranging data on import but that feels a little wrong... There's trade offs to every solution and I'm not sure which is the best.

IlluminatiWave commented 1 year ago

After doing some tests, I really don't know

AssetRipper allows to export files in "raw" (what I thought was the original file). But also in WAV and OGG

When I wanted to extract the fsb (fsbextractor), it tells me that internally it's an OGG file...

But UABE / UAEBA detect the .ab as WAV (which is strange, because if I export it in WAV, it weighs about 20MB when the .ab weighs just under 7MB). image

These asset bundles are located inside

main.[Version of the apk].com.rayark.cytus2.obb\assets\AssetBundles\songdata[Character][Songid].ab

For example

main.55561300.com.rayark.cytus2.obb\assetsAssetBundles\songdata_xenon001_012.ab

And inside each assetbundle (named songdata)

You have several txt files (which are the levels themselves) and the music file that accompanies the

https://youtu.be/5tQUv8boWNI (the video is not mine)

In this video, you can see what the assetbundle is used for.

  1. the txt (it tells the game where it has to put each note to be played)
  2. The audio file (rhythmic background music)

Also the game downloads more of these assetbundles and puts them in the folder android\data\com.rayark.cytus2.obb\files\AssetBundles\songdata[Character][Songid].ab (yes, it's a folder)[MD5].ab (puts the md5 hash of the file as name... A strange implementation).

By the way, I think doing this in batch form is very long term. Allowing to edit audio files one at a time would ensure as few errors as possible (or work on a temporary copy when trying to modify the audio files).

nesrak1 commented 1 year ago

When I wanted to extract the fsb (fsbextractor), it tells me that internally it's an OGG file...

Extract the fsb from what file? I haven't used that tool before, but I'm pretty sure it would take the data in the resource file of the bundle. If you gave it the AssetRipper raw data, it might be the same thing.

But UABE / UAEBA detect the .ab as WAV (which is strange, because if I export it in WAV, it weighs about 20MB when the .ab weighs just under 7MB).

It's not "detecting" anything, these are real paths that are stored just like that in the file. The wav extension in the container does not mean it's currently stored as wav, it means the original file name was wav. As for why it's bigger, the fsb format of course is compressed, whereas wav isn't compressed at all. Internally, it might be stored like an ogg which is why you see it "identify" as ogg even though originally it was wav.

IlluminatiWave commented 1 year ago

Well, ignoring the type of audio file. Have you thought about whether it could be treated as a plain text file to be able to modify the audioclip of an asset bundle?

I don't really know what process you follow to be able to modify the text files inside an asset bundle, but I have the assumption that if it were treated as a text file, the audio file could be replaced by another one.

For example, going back to "songdata_xenon001_010.ab".

If the program tries to interpret "xenon001_010.wav" as a text file, the content would be unreadable to a human, due to the large amount of strange symbols inside it. The idea is that UAEBA deletes all the content of "xenon001_010.wav" and leaves it as a 0-bit plain text, and then reads an external music file as if it were another text file (equally unreadable for us) and pastes the content into the wav inside the asset bundle.

The process could be the same if inside the asset bundle there is only the fsb file, the content would be replaced by another fsb (that can be created with unity or exporting from Asset Ripper) but using the same methodology (of modifying plain text).

Or allow the program to modify the Audioclips one by one (something tedious for batch operations, but it's a step forward).

Obviously to be able to determine this, the program should be able to know if the unity file is compressed (fsb) or uncompressed (wav, ogg, etc), give a message to the user so that he can choose the format for the most successful import.

nesrak1 commented 1 year ago

it could be treated as a plain text file

It can't, that's the whole issue. Unlike textures or text assets, I can't insert data directly into the asset. It has to be read from a separate file.

Getting the format into a compressed form is not an issue, assuming a good library is found for it. Getting the format into a place that can handle multiple edits is an issue.

CreatorsOfficial commented 1 year ago

well creating an fsb is no fun at all

For the audio import a whole lot of questions rise up and many problems plus imperfections. First things first, how to encode the ogg or wav to fsb5 file.(without using Fmod FSBANK as its licensed) Secondly, there is no library to do so yet even thought I tried writing one in C# but the use of too much unmanaged memory made it nearly impossible to write a library in C# let aside the native C++

Suppose there exists a library then audio data is written outside the assetsfile, so you did have to be very careful with things like that for how it would edit the data of a resource/resS or would it create a new resource/resS and what are the odds of doing so

Even if someone managed to make the audio import possible then hats off to his/her work but audio import for UABE C++ can be possible as its C++ and can make use of libraries like ogg and vorbis