search

unexpectedpanda / retool

Retool: a better filter tool for Redump and No-Intro DAT files.
BSD 3-Clause "New" or "Revised" License
355 stars 23 forks source link

System-specific language overrides wrongly affecting results #276

Closed Keiyuu closed 1 year ago

Keiyuu commented 1 year ago

Describe the bug I'm using the PlayStation DAT to produce a Germany region Demos-only set. I noticed multiple Demos are missing from the resulting DAT and it took me a while to figure out the culprit is the system-specific language setting. Since I use English and Japanese as the global setting, I checked "Override global settings" on the languages tab under System settings and used an empty list (to include all potential languages). This results in Retool removing multiple demos:

These are the ones I caught. All of them are in the "Demos" category in Redump. If I uncheck the system override for languages and set global languages to an empty list, they are included.

To reproduce

Expected behavior All Germany region Demos to be included in the DAT

Operating system

Retool edition

Retool version 2.01.1

Additional context Likely unrelated but Retool also removes the different German versions of "Demo One" and only keeps "Demo One (Version 4)" despite them all being different. There might be more occasions but this is what I caught so far. I don't think this behavior is desired since Demo discs are usually all unique and not upgrades of each other. http://redump.org/disc/47282/ http://redump.org/disc/46256/ http://redump.org/disc/1038/

unexpectedpanda commented 1 year ago

Looks like more than demos are getting lost. That's definitely not ideal, I'll look into it.

Likely unrelated but Retool also removes the different German versions of "Demo One" and only keeps "Demo One (Version 4)" despite them all being different. There might be more occasions but this is what I caught so far. I don't think this behavior is desired since Demo discs are usually all unique and not upgrades of each other.

You're correct, the goal is keeping titles that are unique, and yes, it's unrelated to the other issue. The problem here is generalized detection of versions when the naming systems in play are inconsistent or not well followed. There also isn't a concept of tagging build versions in a way that's unique to, say, a variation of a title or something with "version" or "V2" literally in the name.

This is why titles like Mobile Suit Gundam - Version 2.0 and Micro Machines V4 have exceptions in clone lists: to stop them being swept up in the version detection and bundled with the wrong group of titles as different iterations. The PlayStation demos you've mentioned here would need the same treatment.

unexpectedpanda commented 1 year ago

Fixed in 2.01.2.

Keiyuu commented 1 year ago

I just tried the new version 2.01.3 and it's still bugged. Now it's reversed. It keeps the Demos I mentioned above but removes everything else, resulting in only 11 Demos in the DAT.

unexpectedpanda commented 1 year ago

That's odd -- I thought it was working before I pushed out the change. I'll take another look. Perhaps I was looking in the wrong place (like relying too heavily on traces) and system level stuff needs an integration test too 😑

Edit: Confirmed, I'm seeing it too. Looks like I've got more untangling to do.

Edit 2: Here's why I didn't pick it up: I was using command line Retool, which requires you to pass -l to enforce the language filtering. Like an idiot, I didn't, and so was getting all the languages in the output.

unexpectedpanda commented 1 year ago

Okay, one more shot. 2.01.4 should put this one to bed. Let me know if you're still seeing unexpected behavior.

Keiyuu commented 1 year ago

The new version fixed it. Thanks!