search

lmmfranco / nintendo-switch-eshop

Crawler for Nintendo Switch eShop
Apache License 2.0
520 stars 82 forks source link

[Question] How does games / prices data fetching works? #150

Closed alekpentchev closed 4 years ago

alekpentchev commented 4 years ago

Hi! :) Thanks for publishing your app. It looks really nice! :) I was wondering what are the data sources you use for fetching games info and prices? In case of EU and Japan I see some Big N related endpoints but for the US there is this Algolia thing. Did you create this service on your own or is publicallt available? If it's your's then how did you manage to get access to Nintendo data?

Sorry guys for many questions but after doing some research directly on Nintendo sites it was hard to find any reasonable API documentation.

https://github.com/lmmfranco/nintendo-switch-eshop/blob/d9be4fd89e220015b69aec72040eeea1b939155a/src/constants.ts#L13

favna commented 4 years ago

Nintendo manages the algolia based search. The required keys have been found by the simple virtue that Algolia requires those keys to be public (they are called "search keys", as opposed to "admin keys" they can only do searching) and when you search for a game on https://nintendo.com you can see the same keys in the browser devtools network tab / console: image

Algolia is a powerful search provider for API's. The actual data is held somewhere else and Algolia is just the public way to access it.

In a similar fashion to the keys the noa_aem_game_en_us index name you'll find in https://github.com/lmmfranco/nintendo-switch-eshop/blob/a15a7a6bff80e380f6d0566436d4ce18495fda81/src/nintendo-switch-eshop.ts#L59 can also be reverse engineered from the same page:

image

If you want to learn more about Algolia you can visit their site: https://www.algolia.com/ If you want some other pages that use it, there is Twitch (all searches) and Slack (iirc plugin searches at least) as well as the yarn package registry: https://yarnpkg.com/

alekpentchev commented 4 years ago

Wow, thanks! That's a lot of very interesing information. :) And it was clever to find out how Nintendo does it. I know now more than I thought I will so thank you. I'll allow myself to close this question.