KillzXGaming / Switch-Toolbox

A tool to edit many video game file formats
GNU General Public License v3.0
953 stars 154 forks source link

Decompressing .mc files from TOTK #593

Closed ctvicks closed 1 year ago

ctvicks commented 1 year ago

Hi, just wanted to know if anyone knows how to decompress .mc files from TOTK? It seems the models from the game are stored in that file format.

zoalasaurusrawr commented 1 year ago

I've had luck in reverse engineering parts of the file. The magic bytes are MCPK, but the file itself doesn't adhere to the MCPK format used by other games.

Within, you can find each file entry is actually a ZSTD stream, and most can be decompressed using ZSTD once split out from the original .mc file. All that said, I don't see any intelligible model files yet. I've had similar luck splitting .txtg, and while those files do look like the structure you'd expect from a binary stream of color data, they don't turn into anything interesting. Next up, I hope to try a variety of decoding attempts to see if I can get anything out.

KillzXGaming commented 1 year ago

I've had luck in reverse engineering parts of the file. The magic bytes are MCPK, but the file itself doesn't adhere to the MCPK format used by other games.

Within, you can find each file entry is actually a ZSTD stream, and most can be decompressed using ZSTD once split out from the original .mc file. All that said, I don't see any intelligible model files yet. I've had similar luck splitting .txtg, and while those files do look like the structure you'd expect from a binary stream of color data, they don't turn into anything interesting. Next up, I hope to try a variety of decoding attempts to see if I can get anything out.

Do you know how the format is structured? I couldn't make out how the zstd data is being done, like where is starts/ends and where the frames are at. It seems different than usual as it lacks the typical zstd identifier.

zoalasaurusrawr commented 1 year ago

I haven’t gotten that far for .Mc files but I have a bit for txtg!!

https://gist.github.com/zoeysaurusrex/0c8a1ae2e218d1c76959765bf9e9c408

zoalasaurusrawr commented 1 year ago

@KillzXGaming I misspoke here. I’ve been looking at the data so closely for the last week that I mixed up the model files with txtg. The thing I found in the model files are FMSH tags that seem to separate content, but I haven’t gotten further than that.

I suspect that FMSH stands for caFe Mesh much like with FMDL and FVTX.

guighub commented 1 year ago

An update on this? Is there a method to decompress these .mc files yet?

KillzXGaming commented 1 year ago

mc files can be decompressed with zstd using magicless headers, compressed data starts at 0xC. Though due to strings being externally loaded and the FMSH section (stores additionally compressed data that hasn't been figured out yet) we cannot rip models still and will likely be awhile.

Rigodron commented 1 year ago

It's meshcodec compressed. I cant really say how to decompress them, but i have some tracks.

guighub commented 1 year ago

I tried trimming everything before 0x0C on the .mc file then decompressing with ZSTD, but Toolbox still says the decompression failed. What am I doing wrong? I think I have a tool that can import the compressed bfres, or at least it can with the Splatoon 3 ones. I don't know if the format has been updated since.

Rigodron commented 1 year ago

It's normal, you have more things to take into account like the dictionary that must be in the executable, the size of the window, etc. For the moment I'm trying to grab the MeshCodecDecompression thread contained in the got.plt.

KillzXGaming commented 1 year ago

The zstd is magicless so it can't be decompressed with it and the zstd cli tool doesn't support the magicless setting either. It would have to be manually set in code with the official zstd library or some wrapper for it using the setFormat function.

Rigodron commented 1 year ago

The zstd is magicless so it can't be decompressed with it and the zstd cli tool doesn't support the magicless setting either. It would have to be manually set in code with the official zstd library or some wrapper for it using the setFormat function.

Of course, but it's better to go and see how it works than to do it blindly.

DoMx commented 1 year ago

Hi all - I know nothing about code and/or ripping. Good start. I did, however, stumble across a person that has claimed to have ripped TOTK models and posted image 'proof' so I thought I'd share in case their method unlocks anything here.

Thread: https://twitter.com/Avovet3D/status/1660886535418552320

Posting here in case they delete/lock their Twitter:

Is that... helpful?

Rigodron commented 1 year ago

Hi all - I know nothing about code and/or ripping. Good start. I did, however, stumble across a person that has claimed to have ripped TOTK models and posted image 'proof' so I thought I'd share in case their method unlocks anything here.

Thread: https://twitter.com/Avovet3D/status/1660886535418552320

Posting here in case they delete/lock their Twitter:

Is that... helpful?

This isn't bad for those who only want models for other things, however, this technique doesn't allow you to have the native models or a conversion back to the original format.

XDM-Inc commented 1 year ago

Hi all - I know nothing about code and/or ripping. Good start. I did, however, stumble across a person that has claimed to have ripped TOTK models and posted image 'proof' so I thought I'd share in case their method unlocks anything here.

Thread: https://twitter.com/Avovet3D/status/1660886535418552320

Posting here in case they delete/lock their Twitter:

* ok so you know renderdoc right?

* you hook it into ryujinx in vulkan and use an fbx exporter to extract the models

* https://github.com/FXTD-ODYSSEY/renderdoc2fbx

Is that... helpful?

i thought that renderdoc2fbx plugin was abandoned LONG ago, i NEVER got it working on linux OR windows. i just do the old csv method

Odinv24 commented 1 year ago

So i used the recommended addon yet, when exporting the fbx its just empty

CloudOniiKun commented 1 year ago

So i used the recommended addon yet, when exporting the fbx its just empty

Yeah, I've been having the same issue. Not to mention I have to convert it from ASCII to Binary because blender doesn't like ASCII FBX format.

Odinv24 commented 1 year ago

So i used the recommended addon yet, when exporting the fbx its just empty

Yeah, I've been having the same issue. Not to mention I have to convert it from ASCII to Binary because blender doesn't like ASCII FBX format.

yea idk what that person on twitter did or if they just lied cause that specific link i have seen often

KillzXGaming commented 1 year ago

You can now decompress .mc files using https://gamebanana.com/tools/13236

As of the latest version, I've added support for the decompressed .bfres that use external buffer/string data. Editing will not work but may eventually get figured out with some other alteratives.

Odinv24 commented 1 year ago

You can now decompress .mc files using https://gamebanana.com/tools/13236

As of the latest version, I've added support for the decompressed .bfres that use external buffer/string data. Editing will not work but may eventually get figured out with some other alteratives.

how do we extract the textures now

Hayden-Fluff commented 1 year ago

I feel like this issue was closed prematurely, as the application itself doesn't yet support decompressing the files to my knowledge, and from what I can tell there's no way to do this without an actual console, of which I cannot mod my own due to it being too recent of a revision. Are there any other known alternatives that would let me decompress just from the game ROM itself?

KillzXGaming commented 1 year ago

At this time I don't plan to support it. Afaik the plugin supports v1.1.2 and works with Ryujinx aswell.

CloudOniiKun commented 1 year ago

You can now decompress .mc files using https://gamebanana.com/tools/13236 As of the latest version, I've added support for the decompressed .bfres that use external buffer/string data. Editing will not work but may eventually get figured out with some other alteratives.

how do we extract the textures now

If you figure this out, could you give an update? Seems Toolbox can see the textures, but I can't seem to export any. Cheers.

XDM-Inc commented 1 year ago

Brilliant! I got to test this!

AdamK2003 commented 1 year ago

You can now decompress .mc files using gamebanana.com/tools/13236 As of the latest version, I've added support for the decompressed .bfres that use external buffer/string data. Editing will not work but may eventually get figured out with some other alteratives.

how do we extract the textures now

If you figure this out, could you give an update? Seems Toolbox can see the textures, but I can't seem to export any. Cheers.

you'll need to extract the .txtg files in the TexToGo folder, they have the textures, I recommend putting both the .dae files and the textures in the same dir without subfolders and messing around with blender scripts to convert to fbx or something (at least that's what I did) if you wanna actually work with the files

here's the script I used

CloudOniiKun commented 1 year ago

You can now decompress .mc files using gamebanana.com/tools/13236 As of the latest version, I've added support for the decompressed .bfres that use external buffer/string data. Editing will not work but may eventually get figured out with some other alteratives.

how do we extract the textures now

If you figure this out, could you give an update? Seems Toolbox can see the textures, but I can't seem to export any. Cheers.

you'll need to extract the .txtg files in the TexToGo folder, they have the textures, I recommend putting both the .dae files and the textures in the same dir without subfolders and messing around with blender scripts to convert to fbx or something (at least that's what I did) if you wanna actually work with the files

here's the script I used

Much appreciated, got the textures, I was just being dumb and for some reason assumed they were packaged with the mesh data. You're a star!

XDM-Inc commented 1 year ago

You can now decompress .mc files using https://gamebanana.com/tools/13236 As of the latest version, I've added support for the decompressed .bfres that use external buffer/string data. Editing will not work but may eventually get figured out with some other alteratives.

how do we extract the textures now

If you figure this out, could you give an update? Seems Toolbox can see the textures, but I can't seem to export any. Cheers.

I found out a easier way to get textures out from the switch toolbox. Read it HERE pretty much just use switch toolbox to browse that textogo folder and it has all the textures that can be loaded and extracted.

CharlesRBLX commented 1 year ago

ayo i mantaged to port ganon.

tear and forehead thingy werent workin in blender so i was lazy. image

CharlesRBLX commented 1 year ago

its rigged.

MykytaLialka commented 1 year ago

001 dive in this stuff only for Purah references. Ask if you need help)

GreatPepperoni commented 1 year ago

How do you extract the models from the .BFRES files? I've tried using Switch Toolbox, but I havent gotten it to work.

CharlesRBLX commented 1 year ago

its not .bfres, its .mc

Hayden-Fluff commented 1 year ago

You can now decompress .mc files using https://gamebanana.com/tools/13236

As of the latest version, I've added support for the decompressed .bfres that use external buffer/string data. Editing will not work but may eventually get figured out with some other alteratives.

I can also now confirm that this tool works with Yuzu as well, at least with the 1.1.0 patch.