Added NASC and 3DS friends server client

[jonbarrow]

Adds a NASC client and the 3DS friends server proto file/NEX client

Some details about how the 3DS handles online since you mentioned here that you hadn't looked into it

The 3DS handles online a bit differently than the WiiU even though they share some NEX servers. The 3DS and NEX predate NNIDs, and as such the 3DS doesn't rely on them. Each 3DS console generates a new NEX account whenever it cannot find save data for the act system module or friends title. The console will generate the NEX password it would like to use, and request a new account be made with it from the NASC server.

NASC (NAS-CTR) is the successor to the Wii NAS server and carries over a lot of design choices from it. All this server does is take in console details and the requested game details to the /ac endpoint and it returns the location of the server and a token to access it.

The NASC server does NOT send any NEX credentials. When the 3DS generates a new account it stores the password it wants to use and reuses it. Once the 3DS registers a new NEX account during the first connection to the friends server, it also stores the PID the friends server assigns it and reuses that as well. This means that the consoles NEX credentials never hit the wire after an account is created, the 3DS only sends NASC it's PID along with it's console details

Some homebrew like this https://github.com/Stary2001/nex-dissector/tree/master/get_3ds_pid_password can be used to dump the current accounts NEX PID/password

NASC also requires the consoles unique device certificate, MAC address, and an HMAC of the users PID. All of these can be obtained by sniffing the traffic of a console. The PID HMAC can also be found in bytes 66-84 of the friend titles save data (thank you @zaksabeast), though it should be possible to calculate this

[jonbarrow]

Any reason why both the friends protocols are in the same file and not separate? I understand they exist on the same server but having one huge file for them both feels pretty cluttered, especially since they are two very different protocols with no overlap (save maybe GameKey? I think that’s the only thing I found that overlaps between them)

[kinnay]

Nice!

Any reason why both the friends protocols are in the same file and not separate?

Probably just a personal preference. I think we should either have them in the same file or drop the V1 and V2 suffixes. The protocols serve the same purpose, and having them in the same file is less typing (e.g. friends.Mii instead of friends_3ds.Mii). That's why I preferred the both in one file approach. The authentication protocols and the match making protocols are also in one file for this reason. Is that okay for you?

A few more things need to happen before I am merging this:

• Write documentation for nasc.py
• What key is used for pid_hmac? We should calculate the HMAC in our code if possible.
• I would guess that the friend structs all inherit Data. We should investigate this and fix the proto file if necessary.
• The parameters that 'do not matter' should be configurable too. All other clients provide setter methods for their parameters. I think we should do the same in NASCClient.

I would also like to test this against the server before merging it, but I'm currently on vacation. I will probably make a few more changes the code when I'm back. Of course, if you disagree with any of my changes, let me know so that we can discuss it.

[jonbarrow]

Probably just a personal preference

It's also mostly just personal preference on my end, so not a huge deal there

The protocols serve the same purpose, and having them in the same file is less typing (e.g. friends.Mii instead of friends_3ds.Mii)

True they do serve the same purpose and it would be less typing, I was thinking in the sense that it might be unintuitive as it isn't clear which systems protocol is being targeted there

The authentication protocols and the match making protocols are also in one file for this reason. Is that okay for you?

The authentication ones make some kind of sense as they share the same protocol ID and seem largely the same between systems (only real differences seem to be some method renaming and some small structure changes?), where as the 2 friends protocols are 2 separate protocols entirely designed for 2 different systems. I actually hadn't noticed that the match making protocols are in the same file, but that's more understandable in my opinion since they have considerable overlap with structures, and would result in a LOT of duplicate code if split

What key is used for pid_hmac? We should calculate the HMAC in our code if possible.

I agree this should be done in code. I'm not sure what the key is though. As far as I can tell it doesn't get generated on the fly, the client stores the hmac once it has been generated as part of the save data for the friends title, and it pulls from that. I'm not even sure which algorithm is used at the moment, though I'm inclined to think it's md5 seeing as the 3DS NEX friends server sends back a md5 hmac of the assigned PID when registering a new account. This uses the hard-coded key c3ef03044c6937d30e6b179f610ea190, though I have had no luck using this key for this hmac. I can provide you with my PID and hmac if you'd like

I would guess that the friend structs all inherit Data. We should investigate this and fix the proto file if necessary.

I am 100% certain they do but made no changes seeing as it was v0 and thus had no extra data from the structure headers

The parameters that 'do not matter' should be configurable too. All other clients provide setter methods for their parameters. I think we should do the same in NASCClient.

Ahh my bad. I left these alone as they truly did not matter, they are ignored by the server or are left overs from the Wii days. Will add

I would also like to test this against the server before merging it, but I'm currently on vacation

No worries, I've already tested against Nintendo's servers and my own

I'm currently on vacation. I will probably make a few more changes the code when I'm back

Take your time! Enjoy your vacation

[kinnay]

This uses the hard-coded key c3ef03044c6937d30e6b179f610ea190

Where did you find that key? You're probably right in that the hmac is returned by the friend server on account creation, but in that case the key is probably kept secret by the server.

[kinnay]

Do you want to take a look? I made some changes after reverse engineering the friends sysmodule a bit. If you have no further comments I'll write documentation and merge this.

I don't know what makercd and unitcd are supposed to be. I added them to set_title and set_device respectively but I'm not sure if that's correct. I guess we can move them elsewhere if it turns out to be wrong.

Also, in my traffic dump the hmac was only 4 bytes, but your comments mentions a range of 18 bytes. Do you know what's up? Maybe it takes the first 4 bytes of the hmac or something? We should clarify that.

[jonbarrow]

This uses the hard-coded key c3ef03044c6937d30e6b179f610ea190

Where did you find that key? You're probably right in that the hmac is returned by the friend server on account creation, but in that case the key is probably kept secret by the server.

The key is sent by the 3DS in AccountManagement::NintendoCreateAccount as the strKey parameter. I verified this with multiple accounts on multiple consoles and console revisions (3ds, n3ds, 2ds, etc) and they all use the same key. The server uses this key to make an md5 hmac of the assigned PID and returns it in pidHMAC. These 2 hmac's are different though

Just as an example, one of my 3DS accounts has the pid 1877748756, but the uidhmac sent to NASC is 817afa89, much shorter than the md5 hmac

I don't know what makercd and unitcd are supposed to be. I added them to set_title and set_device respectively but I'm not sure if that's correct. I guess we can move them elsewhere if it turns out to be wrong.

These seem to be left overs from the Wii days as they are present in NAS as well, so I'm not entirely sure what they are. The naming convention used in NAS and NASC makes me think they mean "Maker Code" and "Unit Code", based on the fact that gamecd is the games product code. I assume this has something to do with the games publishing, such as identifiers for the developers (maker) and individual title (unit) but I'm really not sure. Adding them to set_title works, though I've only ever observed the values I hard coded so I assumed they were unused

Edit: After checking your changes I see you also came to the same conclusion with the naming convention

Also, in my traffic dump the hmac was only 4 bytes, but your comments mentions a range of 18 bytes. Do you know what's up? Maybe it takes the first 4 bytes of the hmac or something? We should clarify that.

Yes my apologies on not being clear there. The information about the hmac being pulled from those bytes in the save data came from @zaksabeast via Discord, and I had not checked it myself. I assumed it was correct because they also did a lot of reverse engineering of the act and friends system modules, but after checking the save files myself I actually cannot find the hmac there at all. So that seems to be incorrect. Yes, my hmac is also always 4 bytes. I had also assumed it was only taking portions of the hmac sent back from the NEX server but that also doesn't seem to be the case

Do you want to take a look? I made some changes after reverse engineering the friends sysmodule a bit. If you have no further comments I'll write documentation and merge this.

Looks good to me! Sorry I never got around to writing the documentation here, I've been sick with covid and my apartment had a gas leak so I had to evacuate. I only got back home recently

[kinnay]

The key is sent by the 3DS in AccountManagement::NintendoCreateAccount as the strKey parameter. I verified this with multiple accounts on multiple consoles and console revisions (3ds, n3ds, 2ds, etc) and they all use the same key. The server uses this key to make an md5 hmac of the assigned PID and returns it in pidHMAC. These 2 hmac's are different though

Ah, yes, thanks for reminding me. The strKey is derived from the password of the new account if I remember correctly. This is how the client tells the server the desired password during account creation. Weird that it's always the same key for you though. I can't imagine that the password of the NEX account is hardcoded. Will need to do some reverse engineering.

These seem to be left overs from the Wii days as they are present in NAS as well, so I'm not entirely sure what they are. The naming convention used in NAS and NASC makes me think they mean "Maker Code" and "Unit Code", based on the fact that gamecd is the games product code. I assume this has something to do with the games publishing, such as identifiers for the developers (maker) and individual title (unit) but I'm really not sure. Adding them to set_title works, though I've only ever observed the values I hard coded so I assumed they were unused

Yeah, pretty sure it's maker code and unit code, but that doesn't really explain their meaning. I assumed that unit refers to the device (maybe it's the distinction between a dev and prod unit). Maybe it will make sense if we can figure out how the friends sysmodule generates those parameters, but I'm not really good at reverse engineering 3DS software yet.

Yes my apologies on not being clear there. The information about the hmac being pulled from those bytes in the save data came from @zaksabeast via Discord, and I had not checked it myself. I assumed it was correct because they also did a lot of reverse engineering of the act and friends system modules, but after checking the save files myself I actually cannot find the hmac there at all. So that seems to be incorrect. Yes, my hmac is also always 4 bytes. I had also assumed it was only taking portions of the hmac sent back from the NEX server but that also doesn't seem to be the case

Ok, will need to do more reverse engineering then.

Looks good to me! Sorry I never got around to writing the documentation here, I've been sick with covid and my apartment had a gas leak so I had to evacuate. I only got back home recently

Don't worry, I hope you're okay. It's probably better to finish the implementation before writing the documentation anyway.

[kinnay]

Ok figured out the first part by reverse engineering IOS-FPD. Seems like the account is created with the password 'dummy'. The key c3ef03044c6937d30e6b179f610ea190 is generated by MD5-hashing the string 'dummy' 65000 times. I can't believe Nintendo's security is so bad though, so there must be something else going on.

[jonbarrow]

The strKey is derived from the password of the new account if I remember correctly. This is how the client tells the server the desired password during account creation

The password isn’t the same each time, it seems like only the key is hard coded. From what I’ve observed the password doesn’t influence the key at all, though I’ve only checked network traffic not the sysmodule itself here. It may just be different on the 3DS than the WiiU?

Maybe it will make sense if we can figure out how the friends sysmodule generates those parameters, but I'm not really good at reverse engineering 3DS software yet

given that these are reused from the Wii maybe it would be worth asking people from that scene? Wii custom servers are already a thing so they might have already figured this out for us

Ok, will need to do more reverse engineering then.

I’ve reached out to Zak again on discord, we’ve been talking in the Pretendo dev channels. It’s possible I just misunderstood what they said given that the 3DS is new to me as well

Don't worry, I hope you're okay. It's probably better to finish the implementation before writing the documentation anyway.

thank you, I’m doing much better. I agree on waiting until things are finalized

[jonbarrow]

Ok figured out the first part by reverse engineering IOS-FPD. Seems like the account is created with the password 'dummy'. The key c3ef03044c6937d30e6b179f610ea190 is generated by MD5-hashing the string 'dummy' 65000 times. I can't believe Nintendo's security is so bad though, so there must be something else going on.

This definitely doesn’t match up with what I’ve observed. In my testing the password is always random

[kinnay]

I figured out a bit more about the maker and unit code.

The unit code is hardcoded to '2' by the friends sysmodule. According to this page it is always '1' in Mario Kart Wii. So maybe 1 = Wii and 2 = 3DS.

The maker code is probably taken from the NCCH header.

So I believe I was right about the maker code being related to the title and the unit code being related to the device.

[kinnay]

I just ran experiments against the server, and now I'm sure that uidhmac is the same as the pidHMAC that is returned by NintendoCreateAccount.

Are you sure that strKey is used to generate pidHMAC? I can't reproduce the HMAC that is returned by NintendoCreateAccount with the given key. How did you do that?

I still think that it's a secret server-side key.

[jonbarrow]

I just ran experiments against the server, and now I'm sure that uidhmac is the same as the pidHMAC that is returned by NintendoCreateAccount.

Are you sure that strKey is used to generate pidHMAC? I can't reproduce the HMAC that is returned by NintendoCreateAccount with the given key. How did you do that?

I still think that it's a secret server-side key.

You can see my implementation here https://github.com/PretendoNetwork/friends-secure/blob/master/nintendo_create_account_3ds.go, the token is from our NASC server. During testing it would fail if the hmac wasn’t generated using that key, so I was certain that’s what it’s used for. I haven’t touched that method in a while though so I could be wrong? What’s the point of the key otherwise. It’s ALWAYS the same, and doesn’t have anything to do with the password generation (that’s sent to NASC already)

[zaksabeast]

Hey there :wave: I may be able to help provide some info and confirm a few things.

The maker code is probably taken from the NCCH header.

More specifically, the friend sysmodule gets it from FS_ProductInfo.companyCode by calling FSUSER_GetProductInfo.

Also, in my traffic dump the hmac was only 4 bytes, but your comments mentions a range of 18 bytes. Do you know what's up? Maybe it takes the first 4 bytes of the hmac or something? We should clarify that.

Yes, my hmac is also always 4 bytes. I had also assumed it was only taking portions of the hmac sent back from the NEX server but that also doesn't seem to be the case

It's stored as a null terminated utf16. 18 / 2 = 9 characters. 9 - 1 null terminator is 8 characters. 8 / 2 characters per byte = 4 bytes.

I just ran experiments against the server, and now I'm sure that uidhmac is the same as the pidHMAC that is returned by NintendoCreateAccount.

JobNintendoLogin::StepWaitingForCreateAccount (.text + 0x1bc80) copies the pid hmac to the account config structure, so I presume you're correct about it being returned by Nintendo.

[jonbarrow]

JobNintendoLogin::StepWaitingForCreateAccount (.text + 0x1bc80) copies the pid hmac to the account config structure, so I presume you're correct about it being returned by Nintendo.

I ran a few tests against both the official and our custom servers and this didn’t seem to be the case? Even when using our servers, where I control the key used, the hmacs seemed to differ

[zaksabeast]

Oops, something I missed this earlier:

Seems like the account is created with the password 'dummy'.

This definitely doesn’t match up with what I’ve observed. In my testing the password is always random

The password is generated at .text + 0x38c0 by modifying the result of PS_GenerateRandomBytes to be valid utf8 characters.

I ran a few tests against both the official and our custom servers and this didn’t seem to be the case? Even when using our servers, where I control the key used, the hmacs seemed to differ

Just for clarification, are you saying the hmacs differed between multiple accounts or the hmac sent by the server differed from the uidhmac header?

[kinnay]

@zaksabeast Thanks for the clarification. That makes sense.

@jonbarrow I'm ready to accept this pull request. If you give me the green light I'll merge it.

I also made a wiki page btw: NASC Server

[jonbarrow]

Just for clarification, are you saying the hmacs differed between multiple accounts or the hmac sent by the server differed from the uidhmac header?

@zaksabeast Yes in my previous testing this is what I had seen, though this was over a year ago so it seems I misremembered. I just tesed this again with my friends server and uidhmac is taken, in full, from the friends server

In my latest test I used c3ef03044c6937d30e6b179f610ea190 as the hmac key and the pid 1177573236, which results in the md5 hmac 2b7fb0a715159355d75434e9e7c20234. After checking what the console then uses as uidhmac it now sends MmI3ZmIwYTcxNTE1OTM1NWQ3NTQzNGU5ZTdjMjAyMzQ* which is the full 2b7fb0a715159355d75434e9e7c20234 hmac (it didn't stop at 4 bytes, so this is controlled by the server)

I'm really not sure what they are using for the hmac here but it does look like I was wrong, my apologies for that

I'm ready to accept this pull request. If you give me the green light I'll merge it.

@kinnay looks great to me!

[jonbarrow]

I also made a wiki page btw: NASC Server

A few things here:

1) You labeled ingamesn as "Unknown". This is carried over from the Wii/DS days. On the Wii this was used to send the current Mii name. It's unused on the 3ds 2) In the error codes section, it should be noted that Nintendo often sends null as the returncd (not base64 encoded, can usually be triggered by sending requests with missing fields) 3) apinfo is the current wifi access point slot being used by the console (it can have 3 slots with saved access points) 4) The 000F in the User-Agent is not always 000F, it's the current FPD version (if that version changes so does the user agent) 5) fcdcert is not the device certificate, it's the consoles LocalFriendCodeSeed_B 6) passwd is labeled "Unknown", however this is not unknown. This is the consoles requested NEX password for when it wants to make a new account

[zaksabeast]

This isn't significant, but a tiny bit more about ingamesn: it's provided to RequestGameAuthentication (frd:u command 0x28) by the game and is up to 24 bytes in size. Though to Jon's point and what's on the wiki, this isn't something games seem to use, even if it is technically available on a per game basis.

[kinnay]

Thanks! I updated the wiki page.

Is the local friend code seed device-specific? If not, we should probably remove it from set_device().

Which part of apinfo refers to the access point slot? Is it the two digits before the colon? If yes, what is the purpose of the digits behind the colon?

I'm merging this by the way.

[jonbarrow]

The local friend code seed is device specific, yes. And yes it is just the first 2 digits in apinfo. I am unsure what the following digits are for

[zaksabeast]

The function at .text + 0x1348c sets up the apinfo header.

The digits after the colon from ACU_GetNZoneApNumService or ACU_GetConnectingHotspotSubset, depending on the result of ACU_GetWifiStatus.