Feature Request: Full CTRNAND transfer ala Decrypt9

[AnalogMan151]

There are still some features of Decrypt9 not present in GodMode9. For example, in Decrypt9 you can dump the ticket.db System File and then inject it to another console no problem, but if you try to copy the ticket.db file in GodMode9 and paste in on another console, that console will fail to boot.

With the release of NTRBootHax, people are also discovering that doing a CTRNAND transfer with GodMode9 does not revive a console (failure to mount NAND error from Luma) while doing a transfer with Decrypt9 does due to header fixes.

Will GodMode9 receive these last few features from Decrypt9?

[ihaveamac]

select ticket.db and calculate cmac. I thought this was done automatically, but either way that should work.

[AnalogMan151]

Ah, I see. I can confirm it's not automatic, but thanks for bringing it to my attention. If i mount NAND_Full as a FAT image, and copy from there would that work too or would that still need CMAC calculation? Also, is CMAC calculation an option on CTRNAND Transfers too?

[d0k3]

CMAC fixes are done in a CTRNAND transfer, but the CTRNAND transfer, as it is now, is designed to be as non-intrusive as possible (other than the D9 one). Won't work for a completely borked CTRNAND. Leaving this open cause I'm still searching for a solution to this.

Also, what @ihaveamac said. I pondered doing these CMAC fixes automatically, but forcing any compatibility fixes into file operations simply doesn't seem right.

[d0k3]

I took the freedom of chaging this issues title. @AnalogMan151 - would you be willing to test such a full CTRNAND transfer once it is done?

[AnalogMan151]

Sure, I have a spare console I can fuck up.

[d0k3]

Am I right in that you'd actually only need those scripting commands to build what you want on your own?

[AnalogMan151]

What scripting commands?

[d0k3]

Correction, that one scripting command: https://github.com/d0k3/GodMode9/issues/175

As far as I understand, you want some functionality that helps you go from totally borked NAND -> working console, right? My suggestion is, I implement that scripting command, and I'll assist with anything else you'd need. If you got a good script, I'll take pull request to add it to the official ones. Does that sound good?

[AnalogMan151]

Ah, that request was for automated CMAC fixing, which is mostly so we can do things like migrate movable.sed, ticket.db and seedsave.bin to another console via scripts, while this request is about CTRNAND transfers not working how they do in Decrypt9. You can fix all the CMACs manually but that still won't revive a borked NAND like Decrypt9 does. The only thing I see different is the fixing of the id0 folder doesn't seem present in GodMode9.

[d0k3]

Well, that's because such a script would first have to copy a ctrnand_full.bin file over to S:/. then you would go on fixing stuff.

[AnalogMan151]

What’s Decrypt9 have that GodMode9 doesn’t then? Why does Auto Transfer of CTRNAND images in Decrypt9 revive consoles but doing a CTRNAND transfer in GodMode9 doesn’t? I looked through the source for Decrypt9’s Auto Transfer and see it recalculates the id0 and updates it in 1:/data then does CMAC fixes.

In GodMode9 I see the Transfer Image to NAND option overwrites the db files, generates secureinfo_c if needed and then fixes the CMACs but I don’t see a fix of the id0. Is it not needed to do that in GodMode9?

Attempting to fix a black screen at power up with GodMode9 results in a Luma error stating “Failed to mount CTRNAND" while using Decrypt9 seems to work and gets the 3DS booting again.

Is it something else I’m missing or overlooking?

[d0k3]

Alright, you see you now got that fixcmac command at your disposal.

GM9 and D9 doing stuff differently is because of different assumed usage scenarios. D9 can even CTRtransfer a N3DS to 2.1, which is simply not needed anymore (and which has a better tool available, SafeCTRTransfer). The CTRtransfer in GM9 was intended as a pure A9LH only tool, for example used in region switching.

Anyways, yes you can fix it, but you need to know about your usage scenario. Is it completely borked CTRNAND -> usable CTRNAND? Or do you want to preserve as much as possible? I'll help you get this set up in a script, but you've got to test and code teh script itself :).

[AnalogMan151]

I forgot about SafeCTRTransfer... hmmmm. Wonder if that works converted as a firm. I'm totally fine writing the scripts and what not, this all started because some people on GBATemp and reddit are trying to revive long dead consoles and for some reason Decrypt9 isn't launching or throwing errors at the SecureInfo_A file and aborting during the transfers while for others Decrypt9 works fine, so with the release of NTRBootHax and GodMode9 being the new standard ARM9 multi-tool, figured it may be useful to have recovery options for bricked consoles. But a recovery script would do just the same.

[d0k3]

Anyways, let me name two scenarios, then we'll figure something out...

Scenario 1: You got a console that for some reason does not boot to the OS anymore, but you can't pinpoint it. CTRNAND (drive 1:/) is still accessible, and you want to safe as much as possible from there. This is the more difficult scenario.

Scenario 2: Same as scenario 1, or not even the CTRNAND is accessible anymore. You just want to revive that thing.

[AnalogMan151]

I'll start with Scenario 2. Most of the people with NTRBootHax already have a new console and have moved on but would like to revive their old ones. Once I have a framework for reviving them, I can work on backing up personal stuff that can still be accessed for later importing.

[d0k3]

Okay, good. First of all, you know the stuff in essential? There is stuff that you may want to try and save. SecureInfo, movable.sed, LocalFriendCodeSeed, NCSD header, HWCAL0/HWCAL1. Some of that stuff may still be saveable, and if you get especially lucky, that NAND may even contain the GodMode9 essental backup at offset 0x200 in the NAND.

Anyways, starting from blank, you need a ctrnand_full.bin from a donor console of the same type (O3DS / N3DS). I'd choose one that's as clean as possible, but any will work. You also need a sighaxed NCSD header and twln.bin / twlp.bin from a donor console (type doesn't matter here, but same donor console is prefered).

Now, copy over the NCSD header, then fix it with a standard TWL MBR (any nonbroken console has this). Next, copy over twln.bin, twlp.bin (no changes required). Copy over ctrnand_full.bin.

In 1:/, copy over all the essential stuff that you could safe (to the correct places, of course). Afterwards, fix CMACs for movable.sed, everything in 1:/dbs, maybe also 1:/data (that's somewhat optional). If you injected the movable.sed, you also need to fix the id0 of that folder (there's an env var for it), but you can also just do that regardless.

As far as I see, that's it. You may want to split it into an injector and a CTRNAND fix script. And stuff being there or not there makes this take more thought.

EDIT: forgot something. On N3DS you may want to inject a clean sector0x96.bin (it's in S:/). And don't forget about the most important thing... Boot9strap into both, firm0.bin and firm1.bin.

[AnalogMan151]

Now, copy over the NCSD header, then fix it with a standard TWL MBR (any nonbroken console has this).

This is the only part you lost me on. As far as I know, the NCSD header has an encrypted TWL MBR header at the very end at 0x1BE. But it looks like twlmbr.bin is decrypted? Does the encrypt and decrypt commands work on twlmbr.bin?

[d0k3]

The S:/ drives handles encryption transparently. You really only have to copy a good twlmbr.bin there, nothing else. And, yes, the twlmbr.bin file resides in the NCSD header (end of it), but the header.bin has only the encrypted version.

Also, once you got a good rescue script, don't forget to do a pull request :). If you want it included in the official scripts, of course.

EDIT: it may also be easier to ignore saveable files at first.

[AnalogMan151]

Alright, I've got a lot of the script done, tested and working except for one thing. Copying twlmbr.bin causes a BootROM brick. If I copy just nand_hdr.bin it boots but I can't play NDS games. If I restore the original nand_hdr.bin the test console had, everything works good, no need to copy twlmbr.bin when using original nand_hdr.bin.

What am I missing? Why does another console's nand_hdr.bin break TWL firm and why do I get a bootROM brick when trying to fix it with twlmbr.bin?

BootROM error is: EE - NCSD header validation function failed: NCSD magicnum is invalid or RSA verification failed.

EDIT: Okay, so if I copy just the header, it’s signature is valid but the TWL MBR is wrong. So it boots but looses NDS support. I can update the TWL MBR to get NDS support back but that breaks the signature so it doesn’t boot. Is there a step to re-sign the header after fixing the TWL MBR or am I missing something else?

[d0k3]

You missed an important part, you need a sighaxed NCSD, that means you need to inject the correct sighax signature at the correct place (offset 0). You can do this in the NCSD header file or right inside S:/.

Also, some more remarks...

• It may look like you need separate scripts for N3DS / O3DS, but you don't. Just repair the sector0x96.bin regardless. It is required on N3DS and doesn't hurt on O3DS. Same with retail/devkit - don't change the script, change the input files (in that case, different sig required). Also, donor should match target (N3DS / O3DS, retail / devkit).
• It may be an even better idea to just delete the 1:/data/id0 folder after CTRNAND inject. There's only system settings and user data in there, and, well, our only goal for now is to get this back up and running, with as little uncertainty as possible. It will work without that folder, the system will recreate it upon boot.
• As for the potential saveable files, a bit of an explanation. The files mentioned are unique to your console. Loose them, and they are gone forever. We don't care about warranty at this point, but certain stuff may be important for online play.
• If you want a script that takes scenario 1 into account later, I can help with that, too.
• Don't forget, once you have finished and properly tested this, that what you achieved here is worthy of a release. A lot of people will be happy to use that script you did. Maybe even @Plailect will be interested.

[d0k3]

And... because I just thought about it... why not add a little more magic to it and make that script also setup Luma in CTRNAND, then reboot? Meaning, from GM9, you'd be only one script execute away from a fully functional console :).

[AnalogMan151]

Thank you for the clarification. It'll be 2 script executions, since to copy data from the previous id0 to the new one (should their movable.sed not exist or be transferable) I need to execute 2 scripts as the SYSID0 variable is only set at the beginning of running the script (unless you want to make a MNTID0 variable that returns the id0 of a mounted NAND image). I also intend to set everything up with as little SD removal as possible. And yes, I will release it, though once I get this working I'll need new CTRtransfer images, < 11.5 doesn't work on n2DSXL (can't bypass 3D slider setup). I can make the new and old US versions but will need volunteers for EU and JP.

[d0k3]

Sounds great! As I said, you could just delete that 1:/data/id0 folder. I actually think that would be best, cause that way the user would be presented with the initial setup, as you'd expect on a fresh console.

I can think about adding environment vars, but anything that has to be changed at runtime means work and potential for trouble. Also, really looking forward to this. Do you intend to post the instructions somewhere? Or include in the script?

[AnalogMan151]

Verdict is in, the script appears to be a success. Here was my testing environment:

1. n3DS on 11.5 B9S.
2. Downgraded to 9.2 via GodMode9 and 9.2 CTRNANDtransfer.
3. B9S Uninstalled, console boots as stock.
4. Use Soundhax, udsploit and safehax to launch SafeCTRTransfer.
5. Proceed to downgrade to 2.1
6. Use System Settings to upgrade to 11.5 via internet.
7. BRICK
8. Use NTRBootHax to reinstall B9S and Luma3DS. Get error that CTRNAND cannot be mounted.
9. Boot GodMode9, attempt normal CTRNAND transfer
10. n3DS now no longer has a [1:] drive and attempting to mount S:/nand.bin results in a failure message.

It was in this condition that I was able to run my script and upon completion, the 3DS booted into a fresh setup on 11.5, original FriendCodeSeed & DS mode fully working.

Further testing with slightly less broken NANDs (such as system titles deleted) the script is able to copy and import the following data if present: movable.sed, ticket.db, SecureInfo_A/B/C, seedsave.bin, configsave.bin, friendsave.bin, activity log, mii maker miis, streetpass plaza progress, face raiders progress, and AR games progress.

There is also an option to simply revive without trying to save any data.

Right now I only have the files for n3DS USA, though I will make o3DS USA soon, and am working on finding someone with an actual brick to test this out with on GBATemp.

What I would like to ask, is do you know which files need to match and which ones are generic? I see you said that the twlmbr.bin can come from any working console, but how about nand_hdr, twln and twilp? Are those generic or should they come from the same donor console / region as the CTRNAND?

[d0k3]

I'll reply here, then we'll close this and continue elsewhere, alright? IRC (freenode) or GBAtemp come to my mind, just suggest one.

Anyways, fantastic news! I had no doubt you'd pull this off, tbh. If you want me to I can have a look at that script later, too. And maybe I can help improve it by updating the scripting engine.

As for generic files...

• twlmbr.bin - same for all (if not customized / repartitioned)
• nand_hdr.bin - I suggest you do generic ones, blank out the TWL MBR and the sig. There are two kinds, O3DS and N3DS (again, if not customized / repartitioned)
• NCSD signature - two kinds, one for retail, one for devkits
• twln.bin / twlp.bin- unsure, needs experimentation. There are differences per OS version, but no differences between O3DS / N3DS. Unsure about region and devkit/retail differences.
• sector0x96.bin - two kinds, retail and devkit
• ctrnand_full.bin- you already know about that one, right?

You may also have forgotten about the HWCAL0.dat / HWCAL1.dat files - they are important, they contain calibration data specific to your screen.

For twln.bin and twlp.bin I suggest experimentation. EDIT: could also make sense to have a look at 3dbrews title list for this.

[AnalogMan151]

I've added HWCAL into the script. Comparing the SHA for twln.bin and twlp.bin from a o3DS XL 11.5U retail and a n3DS 11.5U retail and got a mismatch, so for now I'm keeping those separate. I zero'd out the signature on the nand_hdr.bin as well as the twlmbr, I need one from another region to verify if they're the same before I make those shared between old and new types.

I have someone who can test the script out on an o3DS and 2DS US retail units, just waiting for the files to upload now. I can do Freenode, what channel, #3dsdev?

[d0k3]

I can guarantee that the NCSD headers only differ in sig and (encrypted) TWL MBR, and then again only between O3DS / N3DS. Doublechecking is good, though :).

As for TWLN.bin and TWLP.bin - they are different, but then they are not. What you really need to do is mount them (in OSFmount, f.e.) and then compare. TWLP.bin is guaranteed to be exchangeable between any system, but I'd use a clean one (with no photos on it).

I'd suggest #Cakey, I also joind #3dsdev now, and you can pm me, too.

Closing this now as (1) the initial issue has been solved and (2) this has spawned a whole new interesting project.

[d0k3]

Oh, wait, let me add this. TWLN and TWLP are different depending on the stuff you have installed / stored there, of course. If you just want images for a clean system, you can most likely use one for all, but if it's about saving stuff....