team-phoenix / Phoenix

A multi-system emulator and library manager designed to be both powerful and easy to use.
http://phoenix.vg
GNU General Public License v2.0
376 stars 40 forks source link

Phoenix database [WIP] #245

Open athairus opened 8 years ago

athairus commented 8 years ago

This issue will define how the new Phoenix database will be organized.

Define the following IDs:

  1. gameID: ID denoting a set of ROMs built from the same codebase for the same platform (e.g. not ports, sequels or expansion packs)
  2. releaseID: ID denoting region-specific release of a particular ROM with unique box art sold over a particular period
  3. romID: ID denoting a single ROM A single gameID will map to a set of releaseIDs. A single releaseID will map to a single romID.

For example:

Rationale: In the end, we want to display to the user the collection of gameIDs that they have valid ROM paths for. In order to select the right region and release we end up showing we must store the user's choice of releaseID for the given gameID. The releaseID gives us all the information we need to display any metadata necessary and we can later look up the stored path to the ROM via the releaseID's romID.

athairus commented 8 years ago

Generating the database

Get source data

Note that this only gets us titles, IDs, filenames, hashes and serials

Get metadata

Note that I am evaluating all of these sites based on what data is available for Super Metroid, a good litmus test for how well the site's categorized classic games.

TheGamesDB has some good quality boxart but seem to only track US releases.

The Cover Project has the best quality boxart I've seen anywhere on the net, far higher-res than what we need, though. Also, most of the cover images are oriented for print (example: SNES games have back, side and front covers in the same image) so they'll have to be automatically trimmed and shrunk to be usable. On the plus side, they track all kinds of regions.

GameFaqs has little to offer in terms of metadata but track not only regional boxart but also release dates. The boxart itself is just big enough for our use.

IGDB is currently in open beta. Doesn't have a lot of useful stuff quite yet.

Giant Bomb has a rich db of releases and release dates. The box art available can be very good but doesn't seem to follow any standard format making automated extraction difficult.

MobyGames has an even richer database! They mark their available box art with lists of countries they came from along with a database of releases by region. Unfortunately, they do not track revisions.

RF Generation looks promising. Tracks releases.

Online Games DatenBank German. Good amount of detail.

athairus commented 8 years ago

releaseIDs and romIDs are already done by OpenVGDB, which is where I got the names from.

athairus commented 8 years ago

Column names must be unique across all databases, along with table names, unless they are IDs. This way, we can easily take advantage of NATURAL JOIN.

All strings must have a language name (ISO 639-1) attached as metadata. For example, the titles table has these columns: titleID, title, titleLanguage. I'm torn on allowing multiple rows with the same titleID but each with different titles and languages. It might add some unneeded complexity.

athairus commented 8 years ago

This is worth considering as we design the schema: http://gamer.ischool.uw.edu/wp-content/uploads/2014/11/LeeClarkePerti_GameMetadataEvaluation_JASISTpreprint.pdf

athairus commented 8 years ago

Console-specific databases:

NES

panzeroceania commented 8 years ago

Console-specific databases:

SNES