bucanero / apollo-ps3

Apollo Save Tool (PS3)
http://www.bucanero.com.ar/
GNU General Public License v3.0
384 stars 23 forks source link

TLOU save corrupted #20

Closed ghost closed 4 years ago

ghost commented 4 years ago
  1. I made a backup to USB of my TLOU saves before I installed CFW.
  2. I then copied them back onto the internal HDD after CFW was done.
  3. I opened the save files in apollo save tool
  4. No selections apart from the two required at the bottom were made.
  5. I selected "Apply & resign" or similar at the top.
  6. It said everything succeeded.
  7. I started up TLOU and it said the save games were corrupted.
  8. I tried to load them but they are not loadable.

Notes:

How do I fix these saves?

Edit: I can attach the files as a .zip here, but I am not sure if they have any of my personal info / ids etc inside them or if it is safe to do so.

bucanero commented 4 years ago

The Last of Us uses a special hash-check, and I didn't had the chance to test if my code was generating that hash correctly. I'll have to review and do some tests with a TLOU save-game to fix it.

If you still have the unmodified files in your USB stick, you can try:

If you don't have the original un-changed files, then you could try resigning/hashing the broken savegame using the Bruteforce Savadata tool on Windows. That should apply the correct hashes back to the files.

There's no need to be signed to PSN, the save-games should work anyways. In this case I'm sure it's related to the unique hash-calculation in TLOU, and the implementation in Apollo.

I think I can get some TLOU file online, it should be enough for testing my implementation.

bucanero commented 4 years ago

Update: I was digging some more information, and found out that TLOU has an additional second layer of encryption in the save-game file.

Bruteforce Data tool has a specific tlou_save_data_decrypter.exe file to handle this unique encryption. Since I can't find any source code for this encryption, I will have to remove the cheats for The Last of us from Apollo, to avoid anyone getting corrupt saves like you experienced.

btw, If anyone has the source/algorithm for this unique encryption and wants to share it, I'll be happy to add it to Apollo.

ghost commented 4 years ago

I had the original saves so I tried again but this time only did the patch & resign and unchecked the two required options from the bottom. It worked!

For me that is all I wanted. But from apollo's perspective the bug is still there and a more future proof fix would be:

  1. Find out how to handle TLoU unique encryption (maybe reverse engineer that closed source code)
  2. Un-select the hash options at the bottom if nothing else is selected, and do not mark them as required.

But that has fixed my problem, thank you for creating this software!

bucanero commented 4 years ago

ok, good to know that the resigning worked 👍

yes, for now I think I'll just remove the cheats from TLOU and any other game that require a second-layer encryption tool to decode the data.

The required tag is actually required when you apply some game-modifier cheat. Probably as you said, it should only get activated when the user selects a cheat. I'll keep it in mind for the next version.

I'll leave this bug open, in case someone wants to help and reverse-engineer the tlou_save_data_decrypter code so TLOU cheats can be supported by Apollo. From what I quickly checked with Ghidra, the TLOU encryption looks very much like the crypt_64bit...() >> https://github.com/RocketRobz/NTR_Launcher_3D/blob/master/twlnand-side/BootLoader/source/encryption.c

again, a proper implementation would require a lot of work & patience, so for the moment cheats for these games will have to be removed.

bucanero commented 4 years ago

Some good news 😄 I reversed the decrypting tool so I'll be able to support those cheats on Apollo. (once I adjust all the code to big-endian)

As I suspected from my first Ghidra look, the encryption algorithm is basically the same used on the Nintendo DS of course, with a different key_table[] and a secret_key = (SH[@2>r62%5+QKpy|g6

A detailed explanation of this symmetric encryption can be found here: http://www.ssugames.org/pluginfile.php/998/mod_resource/content/0/gbatek.htm#dsencryptionbygamecodeidcodekey1

Anyways, later I'll create a quick repo on GitHub and upload the sources, along with the dumped key_table just for reference. This command-line tool should be also useful for non-windows users. 😉

It was a nice challenge, as I wasn't using Ghidra and OllyDbg in a LONG time 😁

ghost commented 4 years ago

@bucanero Congrats! I am still trying to make a VM to build ps3 things.

I wish someone could teach me how to do this stuff.

bucanero commented 4 years ago

if you're using Windows, there are some tutorials on the psx-place.com forums. I know there are also some docker images with the open-source ps3 toolchain, but I never used them. I prefer to build everything locally, also because most of the time there are no binaries for macOS

btw, the pre-compiled Ubuntu binaries worked for you?

ghost commented 4 years ago

Yes, kind of, apart from the freetype2 issue. Thing is, there are like 10 other steps apart from that, so in the end its the same if I download and build the ps3toolchain myself or take that prebuilt, it saves me one step, albeit a CPU intensive step, and one that did not work for me originally.

I built my WSL environment like 5 times, each time trying to get closed to the perfect set of steps. Now I am like, forget WSL, Ill just use a virtual machine, at least that way I can make snapshots up to a certain point which is set up correctly, then if I mess up something Ill reset it. This fear is a side affect of following instructions without understanding. I cannot believe these projects exist since 2012 or earlier and yet there is no decent tutorial out there.

Everything feels like its hacked together, the slightest incorrect change brings the whole thing down as you can see with that issue you logged in PSL1GHT, that is breaking the whole build, which is needed for everything after it.

@bucanero https://github.com/ps3dev/PSL1GHT/commit/d0eea6e024a6e86435136b058bc7aaf1cabb0581 this PR apparently broke the build of the ps3toolchain. What confuses me is this is a PR to PSL1GHT how did it break my build of the toolchain as that is a separate project? Please if you could clarify. I get it, the toolchain is essentially a load of scripts that load other repos and build them.

Also, are you available on discord, or is there some other mechanism I can bug you with questions without having to write comments here on the TLoU save issue things that are completely unrelated :)

bucanero commented 4 years ago

Also, are you available on discord, or is there some other mechanism I can bug you with questions without having to write comments here on the TLoU save issue things that are completely unrelated :)

my discord is bucanero#3673 , but I might not be online every time, so just leave a message and I'll answer when I see it

Also, I'm usually active on the psx-place.com developer forums, so that's another way if you need to ask something.

bucanero commented 4 years ago

Naughty Dog custom encryption support added in c12beb83c4448cdd940c991d1c575171fe89b5fe

bucanero commented 4 years ago

Fixed in v1.2.0

if you have time, please check applying a cheat or modification to your TLOU savegame. (please keep a backup file just in case)