unexpectedpanda / retool

Retool: a better filter tool for Redump and No-Intro DAT files.
BSD 3-Clause "New" or "Revised" License
367 stars 24 forks source link

1Xtreme (USA) PSX #321

Closed pirsir76 closed 7 months ago

pirsir76 commented 7 months ago

Hello,

I am/was looking for a decent 1G1R sorting tool for dat files. I tried retool for the first time and on the first few games sorted with the 1G1R option I noticed it had overlooked 1Xtreme (USA) for PSX.

Here is some info to recreate: Download the PSX dat file from redump: Sony - PlayStation - Datfile (10776) (2024-03-27 17-24-40) Add datfile to retool. Filter by: Region: USA, World Exclusions: Bad Dumps, BIOS and other chips, Demos kiosks and samples, Manuals, Pirate, Preproduction, Promotional, Unlicensed

You will notice 1Xtreme (USA) is missing.

Edit to add: Unselected all the Exclusions and 1Xtreme (USA) was still missing. Disabled the 1G1R filter and 1Xtreme (USA) appeared on the list.

I suppose this means that the 1G1R filter still has bugs to sort out?

unexpectedpanda commented 7 months ago

It's working fine in this case. 1Xtreme is the same game as ESPN Extreme Games, and so gets removed.

Here's the clone list entry in clonelists\Sony - Playstation (Redump).json:

{
  group": "ESPN Extreme Games",
    titles": [
      searchTerm": "ESPN Extreme Games"},
      searchTerm": "ESPN Street Games"},
      searchTerm": "1Xtreme", "priority": 2}
    ]
},

Here's a snippet from the Wikipedia entry:

ESPN Extreme Games (later renamed 1Xtreme due to the ESPNlicense expiring for Sony...)
pirsir76 commented 7 months ago

OK, thanks.

What about revisions? I noticed Retool wasn't cutting out the revisions. I would like only the most recent revision. Is this possible with Retool?

unexpectedpanda commented 7 months ago

Yes, it absolutely chooses the latest revisions by default, which for the most part is automated so long as the base filename is the same.

If a variant has been missed by the automated process, you can file an issue. There are usually three scenarios that cause a miss:

Clone lists can sometimes get out of sync with the DAT files as No-Intro and Redump update frequently (often daily), whereas I release updates about once a month or so. In most cases Retool still catches most title relationships, as things don't change too much over the month, although occasionally new batches of titles come in from one particular source or someone decides to radically change a naming standard, and people notice.

The last time I did a full update for clone lists was about a month ago, so eventually I'll be able to make time to do a refresh.

pirsir76 commented 7 months ago

I can see it chooses the latest revisions, as you said. I will conduct more testing. Please be on stand-by.

I have a master-filter list that I will compare it to. I'll let you know how well Retool performs. :)

pirsir76 commented 7 months ago

@unexpectedpanda I'm currently just viewing the games list produced by Retool in Romcenter. Is it possible to copy the game names Retool produces in the .dat file to a text document? Or from Romcenter?

unexpectedpanda commented 7 months ago

Options > Also output lists of title names from output DAT files. You can optionally supply a prefix and suffix for each name.

If you want to see what it kept and removed, Options > Also output lists of what titles have been kept and removed.

pirsir76 commented 7 months ago

While looking at the PSX games starting with A. I noticed that Retool gave me 2 extra that are not on my master list: Analog Controller Service Disc (Revised) (USA) Analog Controller Service Disc (USA)

Those two titles are not games but in some other category. Is there a filter option to make those two titles not appear?

My filter options: Regions: USA, World Exclusions: BIOS and other chips, Demos, kiosks, and samples, Manuals, Pirate, Preproduction, Promotional, Unlicensed

To add another question: if another duplicate pops up like 1Extreme/ESPN Street Games is there a way I can check the working out of Retool like you did? { group": "ESPN Extreme Games", titles": [ searchTerm": "ESPN Extreme Games"}, searchTerm": "ESPN Street Games"}, searchTerm": "1Xtreme", "priority": 2} ] },

unexpectedpanda commented 7 months ago

Analog Controller Service Disc (Revised) (USA) Analog Controller Service Disc (USA)

They both have a category of Applications. You could choose to exclude those. Alternatively, you could use a post filter.

To add another question: if another duplicate pops up like 1Extreme/ESPN Street Games is there a way I can check the working out of Retool like you did?

That comes from the clone list for the system. In this particular instance, clonelists/Sony - PlayStation (Redump).json.

But you can also select Options > Also output lists of what titles have been kept and removed to see clones that have been assigned after processing. If you're particularly inclined, you can also perform a trace.

pirsir76 commented 7 months ago

"If you're particularly inclined, you can also perform a trace."

So far I am very impressed with Retool, it appears to be passing all my tests. :)

I came across another title that appears on my master list that is filtered out in Retool: Dave Mirra Freestyle BMX (USA) (Rev 1) which is probably a clone of Dave Mirra Freestyle BMX - Maximum Remix (USA). But, I would like confirmation that it is a clone.

Using the "trace" option in Retool seems to be the most basic way to find out if it is considered a clone. I can see the option "trace" in the Debug Options. But how do I actually perform a trace? There doesn't seem to be any instructions. It comes up with a box asking: "Enter a regex string to trace (case insensitive). I'm not too sure where to locate the regex string for a particular game.

Can you help?

pirsir76 commented 7 months ago

One more thing: I think I have found a genuine error from Retool: Game missing: (PSX) You Don't Know Jack (USA) (Disc 1)

Filter Settings: Regions: USA, World Exclusions: Applications, BIOS and other chips, (Demos, kiosks, and samples), Manuals, Pirate, Preproduction, Promotional, Unlicensed

unexpectedpanda commented 7 months ago

A trace is the slowest and one of the more advanced things you can do. It reports step by step what Retool is doing with a particular title that matches a string.

In your case, try entering just Dave Mirra and then process the DAT file. If you don't know regex, avoid punctuation (except apostrophes) in any search strings for now. There's a link in the documentation about where you can learn regex.

The quickest things you can do I've already mentioned, but I'll be more explicit: open clonelists/Sony - PlayStation (Redump).json in a text editor and search for Dave Mirra to see how things are grouped. You will find:

{
  "group": "Dave Mirra Freestyle BMX",
  "titles": [
    {"searchTerm": "Dave Mirra Freestyle BMX"}
  ],
  "supersets": [
    {"searchTerm": "Dave Mirra Freestyle BMX - Maximum Remix"}
  ]
}

Alternatively select Options > Also output lists of what titles have been kept and removed to see clones that have been assigned after processing. This creates a txt file that you can search for Dave Mirra in, where you'll find:

+ Dave Mirra Freestyle BMX - Maximum Remix (USA)
  - Dave Mirra Freestyle BMX (Europe)
  - Dave Mirra Freestyle BMX (Japan)
  - Dave Mirra Freestyle BMX (USA)
  - Dave Mirra Freestyle BMX (USA) (Beta) (2000-08-21)
  - Dave Mirra Freestyle BMX (USA) (Rev 1)
  - Dave Mirra Freestyle BMX - Maximum Remix (Europe)

One more thing: I think I have found a genuine error from Retool: Game missing: (PSX) You Don't Know Jack (USA) (Disc 1)

Yep, you've found a bug to do with conditional region assignment (don't ask). Disc 2 is getting assigned to Disc 1 as a parent because you don't have a specific region it's looking for in your region order (Germany). I'll have to fix that.

pirsir76 commented 7 months ago

@unexpectedpanda "Yep, you've found a bug to do with conditional region assignment (don't ask). Disc 2 is getting assigned to Disc 1 as a parent because you don't have a specific region it's looking for in your region order (Germany). I'll have to fix that." My region order is USA, World. I'm not sure why you put Germany in brackets?

In regards to Retool choosing Dave Mirra Freestyle BMX - Maximum Remix (USA) over Dave Mirra Freestyle BMX (USA) (Rev 1). What is the logic Retool uses in the cases of clones?

unexpectedpanda commented 7 months ago

My region order is USA, World. I'm not sure why you put Germany in brackets?

I'm not getting into the complexities here of conditional region assignment. It's not worth my time. It'll be fixed in the next release.

In regards to Retool choosing Dave Mirra Freestyle BMX - Maximum Remix (USA) over Dave Mirra Freestyle BMX (USA) (Rev 1). What is the logic Retool uses in the cases of clones?

Do a trace and find out.

pirsir76 commented 7 months ago

Hello unexpectedpanda,

I have a question about the functionality of Retool. I have a group of ROMs that I have downloaded. And I have a .dat file created by Retool. Can Retool be used to compare the .dat file and the group of ROMs that I have downloaded? Like can Retool tell me which ROMs I am missing or which ROMs don't belong?

Or would I need another program for that? If so, which program will do that?

Thanks in advance :)

unexpectedpanda commented 7 months ago

It can't, you need a ROM manager like RomVault.

unexpectedpanda commented 7 months ago

At this point I'm locking this thread. For future questions, I recommend reading the documentation, experimenting yourself, or doing a web search.