Proposal to update how the games json is structured

[ja1984]

Hey!

Well done on making the site and building the games database, big ups to you!

I recently started building my own front end based on your datasource and quickly discovered that it could be improved (from a dev. perspective) so I started building my own that would make it a bit easier for devs to use.

As it would be better for the community to have ONE source of information (since it's pretty much the same information :) ) I'm asking if this is something that would be interesting to change to?

The structure in my JSON ( https://raw.githubusercontent.com/ja1984/sogdb/master/data/games.json ) Looks like this

{
  "games": [
    ...
    {
      "expansions": [
        {
          "name": "The Crew 2 - Gold Edition",
          "store_link": "https://stadia.google.com/u/1/store/details/cc97434908874852a6705d255a605dc8rcp1/sku/1e4107605f83447fa8e04e0abdc578b3"
        },
        {
          "name": "The Crew 2 - Deluxe Edition",
          "store_link": "https://stadia.google.com/u/1/store/details/cc97434908874852a6705d255a605dc8rcp1/sku/ec30180d14de41fcaec12d442b81e248"
        }
      ],
      "countries": ["united states", "canada", "united kingdom", "france", "germany", "spain", "italy", "netherlands", "denmark", "norway", "finland", "sweden", "belgium", "ireland"],
      "description": "In The Crew 2, take on the American motorsports scene as you explore and dominate the land, air, and sea of the United States in one of the most exhilarating open worlds ever created.  With a wide variety of exotic cars, bikes, boats, and planes to choose from, experience the unbridled thrill and adrenaline-pumping excitement of competing across the entire USA as you test your skills in a wide range of driving disciplines. Record every heart-pounding, white-knuckle, moment and share them with the simple push of a button - fame is yours to take! Play with up to 7 friends online.",
      "genres": [
        "racing"
      ],
      "image_slug": "the_crew_2",
      "slug": "the-crew-2",
      "languages": ["english", "french", "italian", "german", "spanish", "breton", "russian", "japanese", "arabic", "polish", "dutch", "kanuri", "chinese"],
      "name": "The Crew 2",
      "game_modes": [
        "single player",
        "online multiplayer",
        "online co-op",
        "competitive"
      ],
      "rating": 66,
      "released": "2020-03-24T23:00:00.000Z",
      "resolution": "unknown",
      "store_link": "https://stadia.google.com/store/details/cc97434908874852a6705d255a605dc8rcp1/sku/94db678cbb6342c3bc0e58f71130b3fb"
    },
    {
      "expansions": [],
      "countries": [],
      "description": "Interact with the words inscribed in the pages of a young girl’s diary to solve challenging puzzles, tackle unique platforming segments and discover an emotional tale set across the vibrant world of Estoria in Lost Words: Beyond the Page. Personal diary entries belonging to a young girl named Izzy will unfold as you explore a fantasy land where words hold immense power. Featuring an immersive story crafted by renowned games writer Rhianna Pratchett, Lost Words: Beyond the Page will mesmerize players with its unique blend of gameplay mechanics and striking watercolor aesthetic. Driven by the player’s curiosity, the world of Estoria and the power it holds will serve as the engine that will propel Izzy through this deeply personal narrative, resulting in a truly memorable experience.",
      "genres": [
        "adventure"
      ],
      "image_slug": "lost_words",
      "slug": "lost-words-beyond-the-page",
      "languages": [],
      "name": "Lost Words: Beyond the Page",
      "game_modes": [
        "single player"
      ],
      "rating": -1,
      "released": "2020-03-26T23:00:00.000Z",
      "resolution": "unknown",
      "store_link": "https://stadia.google.com/store/details/d334bea27ee64ef5a3a9bb14b1d6a88ercp1/sku/ab6a5a0d448d4c7aaed8dfa91ad6f4b9"
    },
    ...
  ],
  "pro_games": [
    {
      "month": "04-2020",
      "games": ["spitlings", "serious-sam-collection", "stacks-on-stacks-on-stacks"]
    }
  ],
  "updated": "2020-04-07T07:42:58.833Z"
}

So the biggest change is that the information for each game is not just an array, and the properties have been broken down so that you can use the store link without the entire link. All dates have been changed to a correct UTC format.

Only issue I can see with a big change like this is that current integration will break until they update to support this. It should however be possible to add the current implementation to this since your object is called data.

[nilicule]

I'd be in favour of updating the structure, but this indeed would be a breaking change so it's probably wise to pull @Mafrans into this discussion as well so we could include things like the uuidmap that we've recently added.

As for the structure, I'm not sure the separation between Pro and regular games is entirely necessary - they're prone to changing frequently and unannounced.

Splitting the values up in an array isn't a huge issue for me, just a matter of changing the UI to parse this correctly.

@Mafrans, any thoughts on this? I'd be on board with having a gamedb_v2.json and updating two sets of data during the transition, for example.

[Mafrans]

I'm in favor of doing something like this, both me and @soap-less (the StadiaRPC developer) are interested in a more readable and usable API. This should probably be placed in a new branch and slowly rolled out so old abandoned projects won't break.

[chocomega]

I have a couple of suggestions/ideas I'd like to share and the reason behind these:

• Merge gamedb.json and uuidmap.json I find uuidmap.json redundant with just incremental indices and prone to error/omission. Search by uuid and sku should be possible by saving an item's uuid and sku as keys.
• Drop the store url The store url would be derived from the uuid and sku. The file would be slightly smaller without redundant "https://stadia.google.com/store/details/" for each item, keeping only relevant information.
• Make unofficial info as optional entries (i.e. resolution and metacritic score) It will be harder over time to keep track of that unofficial information. We will most likely end up mainly with unknown resolutions so having them optional will avoid "unknown" or "" values.
• Split the resolution and framerate as int Easier to sort and order (e.g. 2160 instead of "4K").
• Add missing info (e.g. publisher, developers, etc...) For the sake of completeness and being as close as possible to the store. Useful for search requests.
• Add dates (start-end) for Pro games where applicable A list is available here https://www.reddit.com/r/Stadia/wiki/progameslist
• Add the sku type (i.e. "game", "add-on" or "bundle" if we include those in the db) Just an idea, it might be a big task to keep track of bundles and add-ons.

An example of how it could look like:

{
   "data":{
      "879d82512f5441829250ead1aa678a70rcp1":{ // uuid
         "8ca5e97f091e4f7fbfcf8069ba850b1f":{ // sku nested in uuid
            "name":"Metro Exodus",
            "type":"game",
            "released":"2019-11-18T23:00:00.000Z",
            "imgSrc":"images/posters/metro_exodus.png",
            ...
            // dates where applicable
            "proStart":"2020-02-01T23:00:00.000Z",
            "proEnd":"2020-04-01T23:00:00.000Z",
            // unofficial info, optional entries
            "metacriticScore":81,
            "resolution":2160,
            "framerate":30
         },
         "6ad6555933ee4c34a6ad1d2d4fdf69aa":{
            "name":"Metro Exodus - Sam's Story",
            "type":"add-on",
            "released":"2020-02-11T23:00:00.000Z"
         }
      },
      "15a067411b8d4429b0954434d8f05d48rcp1":{
         "7ed521aacf2a44168fb99cd1e13472ad":{
            "name":"Stacks On Stacks (On Stacks)",
            "type":"game",
            "released":"2020-04-01T23:00:00.000Z",
            "imgSrc":"images/posters/stacks_on_stacks.png",
            ...
            "proStart":"2020-04-01T23:00:00.000Z"
         }
      }
   }
}

[nilicule]

This seems like a workable structure to me

[ja1984]

I have a couple of suggestions/ideas I'd like to share and the reason behind these:

• Merge gamedb.json and uuidmap.json I find uuidmap.json redundant with just incremental indices and prone to error/omission. Search by uuid and sku should be possible by saving an item's uuid and sku as keys.
• Drop the store url The store url would be derived from the uuid and sku. The file would be slightly smaller without redundant "https://stadia.google.com/store/details/" for each item, keeping only relevant information.
• Make unofficial info as optional entries (i.e. resolution and metacritic score) It will be harder over time to keep track of that unofficial information. We will most likely end up mainly with unknown resolutions so having them optional will avoid "unknown" or "" values.
• Split the resolution and framerate as int Easier to sort and order (e.g. 2160 instead of "4K").
• Add missing info (e.g. publisher, developers, etc...) For the sake of completeness and being as close as possible to the store. Useful for search requests.
• Add dates (start-end) for Pro games where applicable A list is available here https://www.reddit.com/r/Stadia/wiki/progameslist
• Add the sku type (i.e. "game", "add-on" or "bundle" if we include those in the db) Just an idea, it might be a big task to keep track of bundles and add-ons.

An example of how it could look like:

{
   "data":{
      "879d82512f5441829250ead1aa678a70rcp1":{ // uuid
         "8ca5e97f091e4f7fbfcf8069ba850b1f":{ // sku nested in uuid
            "name":"Metro Exodus",
            "type":"game",
            "released":"2019-11-18T23:00:00.000Z",
            "imgSrc":"images/posters/metro_exodus.png",
            ...
            // dates where applicable
            "proStart":"2020-02-01T23:00:00.000Z",
            "proEnd":"2020-04-01T23:00:00.000Z",
            // unofficial info, optional entries
            "metacriticScore":81,
            "resolution":2160,
            "framerate":30
         },
         "6ad6555933ee4c34a6ad1d2d4fdf69aa":{
            "name":"Metro Exodus - Sam's Story",
            "type":"add-on",
            "released":"2020-02-11T23:00:00.000Z"
         }
      },
      "15a067411b8d4429b0954434d8f05d48rcp1":{
         "7ed521aacf2a44168fb99cd1e13472ad":{
            "name":"Stacks On Stacks (On Stacks)",
            "type":"game",
            "released":"2020-04-01T23:00:00.000Z",
            "imgSrc":"images/posters/stacks_on_stacks.png",
            ...
            "proStart":"2020-04-01T23:00:00.000Z"
         }
      }
   }
}

I like it! I just have a few thoughts.

Make unofficial info as optional entries (i.e. resolution and metacritic score) It will be harder over time to keep track of that unofficial information. We will most likely end up mainly with unknown resolutions so having them optional will avoid "unknown" or "" values.

I think it's better to have them optional in a sense that they are empty, so that as a dev. you can always rely on the data structure always being the same.

Also I think having the list as an array is a better option as it makes it easier to iterate.

I believe the game should be the "parent" object and stuff like add-ons, extensions should be children of that. That way you will not have to loop the whole list and filter out the games (if you want to show only games)

I would invision something more like:

{
   "data":[
      {
         "name":"Metro Exodus",
         "uuid": "879d82512f5441829250ead1aa678a70rcp1",
         "sku": "8ca5e97f091e4f7fbfcf8069ba850b1f",
         "released":"2019-11-18T23:00:00.000Z",
         "imgSrc":"images/posters/metro_exodus.png",
         ...
         "proStart":"2020-02-01T23:00:00.000Z",
         "proEnd":"2020-04-01T23:00:00.000Z",
         "metacriticScore":81,
         "resolution":2160,
         "framerate":30,
         extras: [
            {
               "uuid": "6ad6555933ee4c34a6ad1d2d4fdf69aa",
               "name":"Metro Exodus - Sam's Story",
               "type":"add-on",
               "released":"2020-02-11T23:00:00.000Z"
            }
         ]
      },
      {
         "name":"Stacks On Stacks (On Stacks)",
         "uuid": "15a067411b8d4429b0954434d8f05d48rcp1",
         "sku": "7ed521aacf2a44168fb99cd1e13472ad",
         "released":"2019-11-18T23:00:00.000Z",
         "imgSrc":"images/posters/metro_exodus.png",
         ...
         "proStart":"2020-02-01T23:00:00.000Z",
         "proEnd":"2020-04-01T23:00:00.000Z",
         "metacriticScore":-1,
         "resolution":-1,
         "framerate":-1,
         extras: []
      },
   ]
}

[nilicule]

Couple of things on the above structure:

I'm not sure picking -1 as a rating value when there's no rating present is a good idea. Probably wiser to pick NULL as a value there, since the value is actually missing.

When it comes to the Pro periods, are we sure that they'll only be offered once? I can see Google offering Pro discounts more than once, in which case placing those Pro time periods in an array is a better solution. Keeping track of these will be a hassle though.

[ja1984]

Actually, after giving this some thought during the weekend, I think we should separate the games themselves and stuff like pro deals (and maybe prices if that ever is added)

[Mafrans]

Keeping track of sales seems like a lot of work for what it's worth, I would recommend against that.

Using a heavily standardized system seems like the way to go. For easy development, two options strike me as most effective.

1. Every key is always available, though their values may be null.
2. There are no null values, instead keys with null values are left undefined.

In JavaScript, either of these are good, but to support other languages and implementations of json (for example, Java's org.json library) the first alternative seems best.

[soap-less]

Keeping track of sales would be difficult, but it would definitely make for a cool feature on one of the projects I'm working on. Depends completely on the amount of work the author wants to put in. I think having Null values would be better for Java, but I don't imagine anyone's going to be using Java around here

[ja1984]

Keeping track of sales seems like a lot of work for what it's worth, I would recommend against that.

Using a heavily standardized system seems like the way to go. For easy development, two options strike me as most effective.

1. Every key is always available, though their values may be null.
2. There are no null values, instead keys with null values are left undefined.

In JavaScript, either of these are good, but to support other languages and implementations of json (for example, Java's org.json library) the first alternative seems best.

I vote #1 here.

Yes, the games list should be the core information for the game.

[nilicule]

Combining a few of the above suggestions, the following seems like a workable structure:

{
   "data":{
      "879d82512f5441829250ead1aa678a70rcp1":{ // uuid
         "8ca5e97f091e4f7fbfcf8069ba850b1f":{ // sku nested in uuid
            "name":"Metro Exodus",
            "released":"2019-11-18T23:00:00.000Z",
            "imgSrc":"images/posters/metro_exodus.png",
            "posterSrc": "images/posters/webp/metro_exodus.webp",
            // type will always be 'game' for the main entry - do we really need this?
            "type": "game",
            // genre of game stored in array 
            "genre": [
                "Action","Adventure", "Role-playing game", "Arcade"
            ],
            // Pro deals - we can leave them empty for now, but at least allow for it in the future
            // will be an array of Pro periods
            "pro": [
                {
                    "proStart":"2020-02-01T23:00:00.000Z",
                    "proEnd":"2020-04-01T23:00:00.000Z",
                },
                {
                    "proStart":"2020-01-01T23:00:00.000Z",
                    "proEnd":"2020-01-20T23:00:00.000Z",
                }
            ],
            // all add-ons go here
            "extras": [
               {
                  "uuid": "6ad6555933ee4c34a6ad1d2d4fdf69aa",
                  "name": "Metro Exodus - Sam's Story",
                  "type": "add-on",
                  "released": "2020-02-11T23:00:00.000Z"
               }
            ],
            "metacriticScore": null,
            "resolution":2160,
            "framerate":30
         }
      },
      "15a067411b8d4429b0954434d8f05d48rcp1":{
         "7ed521aacf2a44168fb99cd1e13472ad":{
            "name":"Stacks On Stacks (On Stacks)",
            // rest of game info
         }
      }
   }
}

Am I missing something obvious?

I think having Null values would be better for Java, but I don't imagine anyone's going to be using Java around here

@soap-less actually, ideally we want the JSON to be pulled by many different sources. I'm thinking about creating a mobile app that'll let you easily parse and filter the current set of games - it's useful as long as Google won't let us search by things like genre.

[chocomega]

If we stick with maps and uuid and sku as keys, for consistency I think the extras section should be:

            "extras": {
               "6ad6555933ee4c34a6ad1d2d4fdf69aa": { // sku as key instead of value
                  "name": "Metro Exodus - Sam's Story",
                  "type": "add-on",
                  "released": "2020-02-11T23:00:00.000Z"
               }
            }

I'm fine with both approaches (exclusively maps or exclusively arrays as proposed by @ja1984) and actually I also like the array approach, not necessarily because it's easier to iterate but because it explicitly tells what the value is (ie. "sku": "6ad6555933ee4c34a6ad1d2d4fdf69aa"). One downside with arrays only though, the dev is responsible for building the map of uuid themself if they need it (e.g @Mafrans).

"type": "game" is indeed superfluous as we make a distinction between the main entry and the "extras" as a child attribute (however they're on the same hierarchical level on the store).

If I were nit-picky, I'd say rename "proStart","proEnd" to "start","end" as it's already in the array named "pro".

Other than that it looks good to me!

[chocomega]

Any update on this? I will gladly help convert the current json once the new structure is well defined.

[kalanihelekunihi]

As a follow on for this thread, as titles are beginning to publish official accessibility data, it would be useful to add it here in an easily searchable way.