spotpass/constants: Add all countries known according to 3dbrew

[ItzSwirlz]

src: https://www.3dbrew.org/wiki/Country_Code_List

[ItzSwirlz]

Testing to see if these are real: https://npdl.cdn.nintendowifi.net/p01/nsa/YapN7dMun6U6CVPx/thmtop/UY/pt/top ^ replacing 'UY' with these country codes:

'BW',
    'EE', 'LS', 'LI', 'MK', 'ME', 'MZ', 'SZ', 'ZM', 'MR',
    'ML', 'NE', 'TD', 'SD', 'DJ', 'SO', 'AD', 'GI', 'IM',
    'JE', 'MC', 'HK', 'MO', 'ID', 'TH', 'PH',
       ' EG', 'OM', 'QA', 'KW', 'SY', 'BH', 'JO', 'SM', 'VA'

all of these above throw 404s when subbing in the country code

[DaniElectra]

Keep in mind that the "app id" YapN7dMun6U6CVPx is different per region, so an app ID won't support all country codes. Try testing with different app IDs to verify

[jonbarrow]

As Dani said, not all tasks work this way. Some enforce regions, some do not. Those which do may not support all regions. For an overview of how the 3DS BOSS server operates, see Dani's gist docs https://gist.github.com/DaniElectra/ada7ecc930a84432f2045f6fcabac84f

PS @DaniElectra we should move this to our wiki at some point

[jonbarrow]

Since we're updating the regions too we might as well look into the languages. I'm not sure if 3dbrew/wiiubrew has a list of possible languages (probably just the 16 standard ones tbh?). But we could likely brute force a list by just requesting https://samurai.ctr.shop.nintendo.net/samurai/ws/REGION_ID/languages for every possible region. This returns all valid language ISO codes for a region

On that note, @DaniElectra I wonder if the BOSS server also works this way? For example the US region only has en and es supported on the eShop, I wonder if this works for BOSS too?

Edit: No my above statement is not correct. The eShop only supports pt and en as languages for the BR region, but https://web.archive.org/cdx/search/cdx?url=https://npdl.cdn.nintendowifi.net&matchType=prefix&filter=statuscode%3A200&fl=original&collapse=urlkey shows this as supporting more:

https://npdl.cdn.nintendowifi.net/p01/nsa/YapN7dMun6U6CVPx/thmlist/BR/**en**/3369 https://npdl.cdn.nintendowifi.net/p01/nsa/YapN7dMun6U6CVPx/thmlist/BR/**es**/3380 https://npdl.cdn.nintendowifi.net/p01/nsa/YapN7dMun6U6CVPx/thmlist/BR/**fr**/3387 https://npdl.cdn.nintendowifi.net/p01/nsa/YapN7dMun6U6CVPx/thmlist/BR/**pt**/3385

[ItzSwirlz]

I did a basic search on 3dbrew for languages, and I didn't find much other than https://www.3dbrew.org/wiki/BCMA#Region_and_language_codes which is already what we have.

[jonbarrow]

I did a basic search on 3dbrew for languages, and I didn't find much other than https://www.3dbrew.org/wiki/BCMA#Region_and_language_codes which is already what we have.

Nintendo has used a standard set of languages since the Wii. They are used in basically every file which uses translations and always use the same order. You can see them here https://nintendo-wiki.pretendo.network/docs/idbe#languages (only 12 of the 16 are known, the last 4 usually aren't populated or use the default language)

Checking the eShop server would likely get us a full list tbh

[DaniElectra]

I think the languages are fine, I've seen the 12 languages pattern repeated everywhere. In some places space for 16 languages is allocated but the remaining 4 slots are always empty or use default values, so I think they are reserved.

[jonbarrow]

the 12 languages

Currently our list has 13 languages. These were pulled from the logs of our BOSS server, so some client somewhere is requesting a 13th language. There may be more (likely at least the 16 we typically see allocated), so the eShop may still be worth checking

[jonbarrow]

Also once we are confident that we have them all, the TODO comment can be removed. Should probably happen in this PR as it seems to aim to complete these lists

[DaniElectra]

Well, us is listed as a language, which doesn't seem right. Where did this come from?

[ItzSwirlz]

Well, us is listed as a language, which doesn't seem right. Where did this come from?

'color' vs 'colour', that's all I'll say. Also, variants of English - such as calling a jacket a 'jumper' instead of a 'coat'

[DaniElectra]

Yeah, but for US I've usually seen language en combined with country US

[jonbarrow]

Where did this come from?

The logs of the BOSS server. I just ran a regex over all the logs and pulled out the relevant parts. That's where the countries came from too. I did this ages ago and have just been reusing the same list. I did not verify them myself, but some client did request this (whether or not it was a console or not is another story I suppose)

[jonbarrow]

@DaniElectra Our logs have a single instance of us being used as the language and it's from the EUR Miiverse posting app. This only occurs once in the whole 290MB log file, so it seems like a fluke tbh. The BOSS server on WiiU at least seems to accept any 2 character string as the language

https://npts.app.nintendo.net/p01/tasksheet/1/tOaQcoBLtPTgVN3Y/solv?c=US&l=us&mode=attr

[ItzSwirlz]

Built like, 1/10 of the database, scraped to see that there are definitely files/things being returned for a lot of them, I'm not going through every one but they seem to be fine. node scrape crashed though (given, you could argue its an incomplete database, however I restarted it, and now each of the possibly invalid/nonexistant codes all have SEVENSPOT.boss files. Here are ones that likely are correct, since they have more than those files (usually 1.dlp.boss, unless DLP = download play)

• EE
• IM
• VA
• BW
• LS
• ME
• MZ

.. ok the more I run the scrape, the more of these .dlp.boss files show up - so I think everything in the pr is valid, thoughts?

[jonbarrow]

node scrape crashed though (given, you could argue its an incomplete database

What was the stack trace? There is no real "incomplete" database. It just has however many tasks were added to it

and now each of the possibly invalid/nonexistant codes all have SEVENSPOT.boss files. Here are ones that likely are correct, since they have more than those files (usually 1.dlp.boss, unless DLP = download play)

• EE
• IM
• VA
• BW
• LS
• ME
• MZ

.. ok the more I run the scrape, the more of these .dlp.boss files show up - so I think everything in the pr is valid, thoughts?

I feel like you have some confusion as to how BOSS works. The names of the files don't really mean anything special. Every game which uses BOSS registers a task, and these tasks are used to download game-specific data for whatever the game wants to use it for. The names and content of the files only has meaning to the specific game it was made for

I'm not really sure what the intent of this test was and I'm having a hard time understanding what you're trying to propose?

[jonbarrow]

@DaniElectra I ran through the eShop and here's the results:

Valid languages from the eShop:

• en
• es
• it
• nl
• de
• fr
• pt
• ru
• ja
• ko
• zh

This is one less than our current list (excluding us, which seems wrong anyway now and should be removed), which is tw. Much like us, this language does not seem to be requested very often. Our BOSS server log file is 290 MB, and this language only appears 19 times. None of those 19 times seem to be for the 3DS, and 12 of them are duplicates (only 7 unique requests). I'm not sure if tw is also a "bad" language or not, tbh

Also the following countries threw error code 3001 in the eShop API, so I couldn't check if they had languages

• MO
• ID
• TH
• PH
• EG
• OM
• QA
• KW
• SY
• BH
• JO

[DaniElectra]

I think Taiwan is a valid language. Not only does it appear documented in other places, but as an example MK7 was released in taiwan and iirc it had NN too

[jonbarrow]

:+1: Sounds good

@ItzSwirlz 3 things before we can merge this:

1. Remove us from LANGUAGES to complete that list as well
2. Remove the TODO comment in the file
3. Add a comment to COUNTRIES linking back to https://www.3dbrew.org/wiki/Country_Code_List to give credit back. Our comments take the form // * COMMENT rather than just // COMMENT

@DaniElectra anything else?

[DaniElectra]

I don't think anything is missing, looks good to me!