Auto-generatate Game.ini files

[pathartl]

Parsing PBP's seems straight forward, but BIN's and ISO's are a bit different. Parsing out the serial number for discs would be crucial for this.

[alucard73]

I do not know much but why not use the same system as psx2psp?

[pathartl]

As far as I know the PSX2PSP source has not been released.

[filmcriticbeta]

If you can dump the directory structure from the image this becomes trivial - the games main executable file was required to use the GameID as its filename. The bios then used the system.cnf text file ( which in turn was located on the disk root ) to determine its location/parameters/etc.

Anyways does the linux isoinfo utility support bin/cue as well? You can use that with the -f flag to dump the directory structure. The code for that is also open source.

IIRC the one exception to this was Tekken 3 Pal - I recall having to hard code some terrible hack into my ePSXe frontend to get it to correctly pass on the appropriate emulation settings. However this was around 15 years ago, so I can't remember the details - sorry.

[fritztoch]

Would we add a file to this distribution listing every release's metadata, and then populate Game.ini from it when the disc ID is matched? Alternatively, is there an online source for this metadata that would let us mine it?

[mtrivs]

Would we add a file to this distribution listing every release's metadata, and then populate Game.ini from it when the disc ID is matched? Alternatively, is there an online source for this metadata that would let us mine it?

My own thought on how this would be implemented, would be to have an array populated with the release data contained here or here for every possible release. I don't own either of these sites, so I don't know if they would help you in obtaining a data dump, but the data does exist. Then each game would need to be evaluated to determine the game name/serial number, seemingly the way @filmcriticbeta mentioned, where all the data could be correlated into the game.ini file. Likely not the most elegant solution, as you may find that different regions use the same title for the game, but have a different serial number (i.e. Asteroids (US) = SLUS-00773; Asteroids (FR) = SLES-01468; Asteroids (DE) = SLES-01469).

An alternative solution to this issue, might be to make the user responsible for renaming the file to match (exactly) the serial number or game title against one of the two databases linked above. From there, each title could be evaluated based on the filename, rather than having to dive into each file to extract the serial number.

[fritztoch]

I checked a couple of these sites, I don't see any links about some sort of API to search against the database programmatically. I personally favour http://psxdatacenter.com but as they are asking for financial contributions, they may not welcome us scraping their site for data. The list of PSX titles is static: there will (likely) never be another new release (although every site I looked at says their lists aren't yet complete), and a text file containing all the titles' metadata shouldn't be too big. I think we should contact these site owners for permission to do a single export of the data they've collected and store that in a file for inclusion in this package, to minimize traffic/server load for them. If their data gets revised periodically, we could approach them for an update periodically. As far as parsing the BIN for the disc ID: I looked at a couple and they don't seem to keep it in the same place in different discs. Renaming the CUE to match the disc ID sounds like a simpler idea.

[mtrivs]

I did some more research and it looks like there are a few other sites that would provide an API that could be used to build a scraper: https://api.igdb.com/. This site does offer an API for free, but it is limited to 3,000 requests monthly. It may be possible for multiple users to query the DB using the API to divide and conquer, until we have a finalized data dump.

https://github.com/sselph/scraper - Scraper designed for RetroPie, that will hash .bin/.cue files and scrape metadata from theGamesDB.net to output a gamelist.xml. The gamelist.xml can then be parsed to match up the metadata and build the .ini file

[filmcriticbeta]

Having thought about it further, I agree that parsing is probably the wrong approach here. Its reasonable to assume that we are dealing with a finite number of titles, an SQLite database is a more robust solution. As a high level concept:

1. User selects add new game
2. Dialog box is then presented to allow user to select the .cue file. ( Not the bin file as explained below )
3. Validate contents of the .cue file. In particular verify that any .bin file(s) referenced within the cue are present and the filenames match. This is mostly future-proofing, whilst PCSX doesn’t seem to care, retroarch is more sensitive to this sort of thing.
4. Other options can be manually entered via the dialog box, with a search option to perform a SELECT title LIKE sql query on the game title. User can then select from a list of options to populate the remaining fields. Will need to consider multi disk games in some manner.
5. User presses ok, at which point game.ini gets generated, any any file renaming/imports occur.

As an aside one question: is game.ini a bleemcast construct, or a requirement of the classic. If it is the former, it may well be best to drop the game.ini concept entirely, and just read/write to the classic’s sql database directly from a user friendly gui tool. Then perform database validation on device via boot.sh during initialisation.

[mtrivs]

Having thought about it further, I agree that parsing is probably the wrong approach here. Its reasonable to assume that we are dealing with a finite number of titles, an SQLite database is a more robust solution. As a high level concept:

1. User selects add new game

2. Dialog box is then presented to allow user to select the .cue file. ( Not the bin file as explained below )

3. Validate contents of the .cue file. In particular verify that any .bin file(s) referenced within the cue are present and the filenames match. This is mostly future-proofing, whilst PCSX doesn’t seem to care, retroarch is more sensitive to this sort of thing.

4. Other options can be manually entered via the dialog box, with a search option to perform a SELECT title LIKE sql query on the game title. User can then select from a list of options to populate the remaining fields. Will need to consider multi disk games in some manner.

5. User presses ok, at which point game.ini gets generated, any any file renaming/imports occur.

As an aside one question: is game.ini a bleemcast construct, or a requirement of the classic. If it is the former, it may well be best to drop the game.ini concept entirely, and just read/write to the classic’s sql database directly from a user friendly gui tool. Then perform database validation on device via boot.sh during initialization.

This is essentially where I was going with this also. The process to extract the meaningful data from a .bin/.cue file, parse those files for information needed to identify the title, then correlate said data with an existing array/database would likely take some significant processing power and would take a considerable amount of time (dependent on processing power). The time to complete this effort would also be increased, if for example the app is enhanced to grab covers for the games from google. Even if you did use hashing against the .bin/.cue files to identify the title, users might still run into issues if they had combined a multi-disc game to a single .bin/.cue file- since the hash would be different at that point. It almost sounds like you would want a "Custom Game" button, where a user can specify all the metadata, add additional disks, etc. to build the game.ini file in a guided manner through the gui. I was thinking the best way to do this, was similar to what you described, by having the title of the file itself searched against the database title and serial fields. Then you could explain to users that in order to use the app, the files must either be renamed to either the title or serial number, in order for the app to pick up the right metadata. Doing something like this would also make it easier to expand in the future, if ISO and PBP support is being planned, as it would only be looking at file names- as opposed to hashing every file.

Ultimately, PBP support would mitigate a great deal of issues in identifying the release, as the files are encoded with metadata during conversion. On the flip side, some games (namely the FF series) will show issues using PBP, due to the sheer size of the file after combining the multiple disks into a single PBP image. So no matter which way you spin it, there will always be a need for identifying the release title/serial for .bin/.cue files, as PBP won't work for every single release.

To answer your question, I am fairly certain the game.ini file is a Bleem requirement and is not required by the PSC, but it is possible I am incorrect.

[fritztoch]

It almost sounds like you would want a "Custom Game" button, where a user can specify all the metadata, add additional disks, etc. to build the game.ini file in a guided manner through the gui.

This is something that could be implemented first, IMO. We could add the database later, but a front end for manually building the game.ini would still be easier than the current. Add a gadget to Google Image search for the cover.

[pathartl]

I wrote a util that opens up a filestream and reads through the disc image and extracts the disc serial number out. It's pretty quick, uses ~1MB of RAM, and runs in all well under 50ms.

[mtrivs]

I wrote a util that opens up a filestream and reads through the disc image and extracts the disc serial number out. It's pretty quick, uses ~1MB of RAM, and runs in all well under 50ms.

HEY NOW! That sounds much faster than I had anticipated. Thank you for the update and for all of your hard work to make this a reality.

[fritztoch]

This appears to link a "complete" list of all regions PS1 titles,

Just in order to share them with people who are passionate about PSone games (like me) and the ones who need them.

https://appnee.com/psone-all-games-full-list-with-official-numbers/

[filmcriticbeta]

Ah. Performing a string extraction on the bin file will pull the boot parameter line out the system.cnf file without the need to understand the underlying file structure, and the boot parmeter contains the GameID in its path. My hat is off to you good sir, that was genius.

[pathartl]

Note that it only works on ISO/BIN at this time. If we want PBP support, that's going to be very easy as it relies on a specific offset in the header.

[pathartl]

It needs to be cleaned up, but the database API is created as a .NET Web API project. I've deployed it to a temporary DigitalOcean droplet. A client has been written into BleemSync (albeit quickly) and game info will now pull from the 'net based on the serial number extracted from the bin/iso.

Some bug fixes need to be done before this can be considered for release. Namely multiple discs aren't being handled properly.

[pathartl]

Added with 0.4.0.