Scanning doesn't pick up PSP ISOs compressed into CHDs

[CosmicHeron]

Description

When scanning content (either individually or whole dir) it appears Retroarch does not pick up PSP ISOs that have been compressed into CHDs. If CHDs are imported using a manual scan or manually loaded they can then be launched by the PPSSPP core so this is not a core issue.

This behavior was reproduced using confirmed good ISOs that are picked up by the content scan if left in their uncompressed form. I suspect the database does not have checksums for PSP CHDs?

Expected behavior

RetroArch is able to pick up PSP ISOs compressed into CHDs as it does for content for other systems (PS1, Saturn, Dreamcast, etc.).

Actual behavior

RetroArch does not detect PSP ISOs compressed into CHDs.

Steps to reproduce the bug

1. Get a good PSP ISO and compress it into a CHD file using mame-tools' chdman
2. Load up RetroArch, in the updater get the latest database, optionally install the PPSSPP core
3. Attempt to scan the CHD, note it will not be added to the PSP playlist
4. Scan the ISO, note it will be added to the playlist
5. Optionally load the CHD (either manually or by importing it into the playlist using the manual scan) and launch it using the PPSSPP core, note the game will load and start

Version

• RetroArch: 1.16.0

Environment information

• OS: ArchLinux, package installed from the official extra repository

[i30817]

This is possible to implement but basically requires special treatment, and probably only using serial scanning, for a consistency reason.

Although chd has a inbuilt and single match sha1 checksum inbuilt, which you'd think would be able to match games in the database, redump doesn't actually include the chd single checksum except by accident of the dumps using a single file format (in the case of the psp it works, because the psp redump dumps use iso dvds, but in the case of the ps1 for example, you're screwed unless you can code the complicated heuristic to divide tracks and calculate individual track checksums.... and the only example I saw of that didn't work for Dreamcast dumps...)

So what you'll probably get in RetroArch is serial scans... assuming there aren't games with the same serial across platforms, which I wouldn't actually be surprised about.

The situation would be better if redump included a chd sha1 field in its dats. And better in other ways too, because a single amortized checksum is much easier to check for emulators and can't have duplicate tracks for multiple games (like that funny time when RetroArch only checking the checksum of cue files - for "speed" - with tosec Dreamcast dumps named almost all games the same - tosec made the mistake of naming their Dreamcast dumps tracks "track #" which in turn caused almost all cues to have the same checksum and not being able to place games in the same directory... Tosec is silly).

My advice, as always: use the manual scanner, and automatic scanner delenda est.

[i30817]

You mentioned that ra picks up chd ps1 games. This is surprising to me actually, because the above. I don't think anything changed with redump, so maybe ra is already picking up serials in the ps1 even inside chds but not so in the psp?

Checked again and although redump now has a 'total' field when there are multiple tracks, it only includes the crc32 not the (useful for chd) sha1 so it's a hit and a miss. Ra is likely using serials for all ps1 games, but not psp games.

http://redump.org/disc/5414/

Note that there are some serial false positives too (from factory printing errors). For instance I have a PS1 urban chaos (usa) (supposedly SLUS-01091) disc, with the correct checksums, that RetroArch automatic scanner insists in identifying as threads of fate (usa) (SLUS-01019). Because the string for threads of fate was used with it no matter what the internet\redump says about it (theyre lyiiiing)

[CosmicHeron]

So I just re-ran a normal scan on CHDs made from PS1 cue/bins with confirmed redump checksums and RetroArch picked up and added >99% of them to the PS1 playlist. That's why I found it odd it wasn't doing the same for PSP CHDs.

[i30817]

Actually now that I think about it some more, if RA can search the database for 'any' sha1 checksum, then look up the game that sha1 corresponds, that might be enough to scan chds by checksum.

Chd has a field called 'data sha1' that should suffice to find 99% of the psp chds if the data is in the databases. And it often is, it's just that chd doesn't separates data in tracks - but the PS2 and psp dumps don't either. Even only some of the ps1, megacd and saturn games do (those with cd music) and ps1 is already searching for serials anyway.

This idea can't be used for dreamcast though (they all have multiple tracks and redump keeps them apart).

Steps are

1. When the scanner finds a chd
2. If it's not of a platform that the scanner recognizes it should scan by serial (ps1, Dreamcast, platforms where the disc images might have multiple tracks)
3. Pick up the data sha1 from libchd and search it in the databases (with the same tricks it does already to search first in the more likely rdbs).
4. When found, instead of using the track name, use the enveloping game name

I still won't use the automatic scanner because it's just 99%, not all (instead of false positives for hacks, it just won't pick them up with this strategy), but it should appease people using the automatic scanner.