search

Perfare / AssetStudio

AssetStudio is a tool for exploring, extracting and exporting assets and assetbundles.
MIT License
13.12k stars 2.52k forks source link

New Format in Netmarble Games #737

Open Grendor opened 3 years ago

Grendor commented 3 years ago

I've been experimenting with various mobile games, seeing what's compatible and exploring assets out of curiosity. I ran across some odd issues when extracting the more recent updates to certain netmarble games. Specifically I've been trying to acquire the assets from the recent collaborations featured in The Seven Deadly Sins: Grand Cross. Apparently there's a new unity format being used for the more recent files as a full extraction fails to acquire the models and textures, or anything featured in them. However, doing string searches using a program known as Total Commander does in fact reveal that the files are clearly present within the newest update package. I was hoping we could see an update to handle these types of files as I would very much like to acquire the recent Stranger Things collaboration models from this specific game, and hopefully help out others who may be having similar issues with other games using this new format.

SacriGrape commented 3 years ago

This isn't because of a new format (Well technically I guess it is, but if the header is removed, its read fine). For some reason some of the asset bundles have an "exta" header. You can remove this using a hex editor.

I wrote a script that you can use to Automatically remove this header. https://pastie.io/iinceg.ts You need to modify lines 8 and 9 to be your own paths. 8 being the folder that contains the asset bundles (In your case, data/com.netmarble.nanagb/files/bm) and 9 being the folder You also need to make sure that this folder doesn't contain ANY files that are not Compressed AssetBundle files (this means deleting the image folder that is there. I will probably eventually modify the script to actually be written well but I don't care enough if I'm honest. It served its purpose)

If you want to have it just replace the file with the double header with the file with the single header, set fixPath and basePath to be the same. If you wish to do this, you need to put an if statement around appendFileSync that checks to see if the file exists first, if it does, use writeFileSync, and if it doesn't use appendFileSync. This code was written by a sleep deprived me and is incredibly dirty and unoptimized but it works.

I compiled with the --downlevelIteration flag but I'm not sure if I actually need it if I'm honest. (Also that code is TypeScript, use tsc --downlevelIteration ./removeHeader.ts to compile.

Have fun ripping the new Re:Zero collab ;p

Grendor commented 3 years ago

Oh, thanks for the response. I've got the script all set up to work but uh...I'm not entirely sure how to go about executing it. I'm not the best at script-related work as my field is graphic design and 3D modeling. Forgive my ignorance but could you inform me on what I need to run this script you have so graciously provided?

SacriGrape commented 3 years ago

Well first thing, I'm going to assume you are using Windows, so the steps I give will be for it. If you need it for linux, it will be slightly different based on your distro. Ill also just go ahead and give a compiled version of the script. I modified it so that it prompts the person what the locations are when its started. https://pastie.io/raw/uzuyqy

First things first, you need to install Node. This will be used to trigger the script. Go ahead and pickup the LTS version from this page here. https://nodejs.org/en/ You can leave all of the options at default, it will do what it needs to do.

Now that we have node install, we need to get the script. Go ahead and just copy the text from this page and paste it into a text file and rename that file to headerRemoval.js (The name doesn't matter specifically, but naming it that is cleaner for the example)

Open up command prompt and navigate to the folder that the script is in. Run node ./headerRemoval.js You should then be prompted for the files folder and where you want to save the files to. Let me know if you have any more questions

SacriGrape commented 3 years ago

Bit of an update, not really related to the script but I'm not actually able to find the Re:Zero models. Ran the script on com.netmarble.blahblah and the models just were not there. I also ran it on the APK data itself but as expected, the models are not there either. At the moment my only guess is that it fetches them from online, but the file count did change between updates so I don't think thats the case

Grendor commented 3 years ago

I've done some deep searching through the files from the recent update. With specific strings I've determined that the Re:Zero files ARE all present,, but I have yet to figure out why they won't read, with or without the script. I'm having a hard time figuring out which of the files contain actual models or just referenced strings. There's odd differences in some of the names, such as Petelgeuse being referred to in the files as 'Petelgius'. From what I can tell, the character files are prefixed with 'herorezero' but due to compression it's hard to find a solid string to determine which archives actually contain them.

SacriGrape commented 3 years ago

If you can find a file that should contain ANY if the rezero files I would love to look at the hex for it and try to figure out what's up with it.

Grendor commented 3 years ago

I found a few that clearly have some relevant files from it in them. I'm not sure exactly what said content is, as it's sort of difficult to discern from how 'broken apart' the strings are. Here's what I could find thus far. https://www.dropbox.com/s/g7a67mi6molncob/7DSRZ.zip?dl=0

SacriGrape commented 3 years ago

Alright, out of the files you sent here is what I found in the files using UABEA

05D4FD711F285F2269F07C1840AB06B0 - This is a story file that has some random Re:Zero subfolder in it. It seems to contain a Re:Zero cutscene (Or multiple of them).
4FBDA0516B8639996578D25A71EBC6AC - This is assets for the Re:Zero bingo
865DE492D16D890CB663D279F4594EFB - This one is confusing, my guess is Re:Zero is mentioned inside of one of hte many MonoBehaviors in this file.
C28B1F3463400FBECFDD88261A8FA352 - Seems to contain files for story, likely just another Re:Brtiania story file.
D99FE9DA4E3B307BAA7C7B2B3F302AEB - Looks to be the file for characters using associations.

Some other interesting things:

  1. Besides the double header, the hex looks normal and there isn't a read issue on using them in AssetStudio or UABEA
  2. For some reason AssetStudio is just missing some assets that UABEA is.

I'm going to make a script that scans every file for the string "rezero" and have it copy them to a folder.

Grendor commented 3 years ago

I hope we can work this out one way or another, I want to have a complete archive of the hero models and such, hahaha. Let me know what your script turns up. I've been using 'Total Commander' to do my string searches and it's...not the greatest at hex.

SacriGrape commented 3 years ago

Thankfully string compression doesn't really exist, anyway I did the search (took ages because I couldn't just read the first 100 bytes and move on like I did for replacing the header) and I found 10 ReZero files (Not nearly as many as I thought there would be so I think maybe I did something wrong?) I'm not really sure what the issue with them is. Also I was unable to find any hero_rezero strings in there so I feel like I am reading this incorrectly. I didn't search specifically for Petelgius or hero_rezero but I feel like rezero would be in them as well. Also of the files you sent, these are the ones I found not there. Not sure if you found them or not but

cut/ED6ED4F21C442A0AB731C6B9D55CEE56
jas/5E3EB3B62931DF1D7CE04CB18D00D1D0
jas/31B7893EB238F7FA2886C789E74ED959
jas/2993D03FB2E8FB1287308D09FC9C4A93
jas/C7D26D049C22C3F1335A2005B580590B

Also not sure if it really matters but the Japanese version seems to have whatever this issue is as well even though that collab happened before the ST collab in the global version and their ST file are findable.

Grendor commented 3 years ago

This is very bizarre. I know the files have to be there, it's never been any different. Also I made a slight error. It's actually 'Petelguis' and not 'Petelgius'. My methods of searching are rather haphazard so I think you'd be more capable of doing it since you have more knowledge in hex and scripting and such. Sorry I'm not much help. All character files are in the 'm' folder, is what I know. 'Cut' is for cutscene information and 'jas' is just the japanese translations.

Grendor commented 3 years ago

Found them. Apparently everything is just prefixed 'herokon' followed by a color. I have no idea why. But they're there. I went through all of the icons in the game one by one and found what they referred to. So yeah, sneaky sneaky.

SacriGrape commented 3 years ago

I'm curious why they would do that, does the color have any meaning to what the character is?

Grendor commented 3 years ago

Well, the ones I could make a connection with is 'hero_kon_white', which is the 'White Whale', and 'hero_kon_green' which is Petelgeuse, and he has green hair so maybe it's something like that?

SacriGrape commented 3 years ago

Yeah the naming does seem related to their color. I wonder why this is the only collab to do that, plus "Kon" doesn't make sense. It doesn't seem to be the japanese name for the anime either so I'm confused why it was chosen.

Grendor commented 3 years ago

Huh, now I'm having problems extracting some of the hero illustrations, specifically that of Excalibur Arthur, a recent release. What's more is I can't seem to find a string for it in any of the current files despite there being the artwork available in-game already...I really try to get all the illustrations for my collection.

SacriGrape commented 3 years ago

I'll look through it when I get the chance, maybe it's the Japanese name for him or something

On Wed, Jun 16, 2021, 1:50 PM Grendor @.***> wrote:

Huh, now I'm having problems extracting some of the hero illustrations, specifically that of Excalibur Arthur, a recent release. What's more is I can't seem to find a string for it in any of the current files despite there being the artwork available in-game already...I really try to get all the illustrations for my collection.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/Perfare/AssetStudio/issues/737#issuecomment-862627097, or unsubscribe https://github.com/notifications/unsubscribe-auth/AO3ZLAQ23JWR3GWH5Y2AZBLTTDXAZANCNFSM44C3SKRA .

Grendor commented 3 years ago

Sounds good. Hope you can figure it out, I can't find it anywhere!

Grendor commented 2 years ago

Hey, with the new PC release of this game, there's apparently some issues with the files. The node script you developed errors out with an allocation error no matter what I do, so either the PC format is different or there's an excessive number of files...

Grendor commented 2 years ago

I'll look through it when I get the chance, maybe it's the Japanese name for him or something … On Wed, Jun 16, 2021, 1:50 PM Grendor @.***> wrote: Huh, now I'm having problems extracting some of the hero illustrations, specifically that of Excalibur Arthur, a recent release. What's more is I can't seem to find a string for it in any of the current files despite there being the artwork available in-game already...I really try to get all the illustrations for my collection. — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#737 (comment)>, or unsubscribe https://github.com/notifications/unsubscribe-auth/AO3ZLAQ23JWR3GWH5Y2AZBLTTDXAZANCNFSM44C3SKRA .

Hey there. Uh, I happened to lose the compiled node script you made for auto-editing these files, but your link no longer works. Any chance you still have it somewhere?