search

unexpectedpanda / retool

Retool: a better filter tool for Redump and No-Intro DAT files.
BSD 3-Clause "New" or "Revised" License
357 stars 24 forks source link

GBA Parent/Clone mixed with compilations issues #20

Closed katarak closed 1 year ago

katarak commented 3 years ago

I feel like most of this will be a bug, but perhaps there are also some issues in the clone list. On GBA many of the compilations are in the dat, despite

  1. Games being available outside of the compilation
  2. Having a compilation in "English"

I generated a dat with (-bcdfrs) Nintendo - Game Boy Advance (1,696) (20201229-143743) [1G1R] (-bcdfrs) (Retool 2021-01-09 11-04-33).txt

For example. 2 Games in 1 - Monsters, Inc. + Finding Nemo (USA) is excluded from the dat because both of those games are available as single titles. However all other games listed as clones are in the dat, probably because the "english" was excluded.

    "2 Games in 1 - Monsters, Inc. + Finding Nemo": [
        ["2 Games in 1 - Die Monster AG + Findet Nemo", 1],
        ["2 Games in 1 - Monsters & Co. + Alla Ricerca di Nemo", 1],
        ["2 Games in 1 - Monsters en Co. + Finding Nemo", 1],
        ["2 Games in 1 - Monstres & Cie + Le Monde de Nemo", 1],
        ["2 Games in 1 - Monstruos, S.A. + Buscando a Nemo", 1]
    ],

2 Games in 1 - Finding Nemo - The Continuing Adventures + The Incredibles (USA) is not listed in the compilation sections. Also the single title is "Incredibles, The (USA)" if that could also cause issues.

2 Games in 1 - Golden Nugget Casino + Texas Hold 'em Poker (USA) is not listed in the compilation sections either, but is found in the renames section. Result is that the compilation and the single titles are in the dat.

In the end I think the real bug is that clones are not excluded if the compilation is excluded, and they should. The other examples are probably just clone list edits. Finding the issues, however, is difficult because of the noise created by that first bug.

I'm happy to do a pass on all the compilations that still show up after the bug is fixed.

unexpectedpanda commented 3 years ago

Ah compilations are really quite thorny. This won't make 0.87, but I'll revisit it when it's time for another update.

unexpectedpanda commented 3 years ago

Okay, finally put aside the time to dig into this. Yep, there are definite problems here that I'd like to solve for 0.88.

You've correctly pointed out that I've missed individual titles where the article has been swapped around in comparison to the compilation. For example, Monster AG, Die (Germany) vs 2 Games in 1 - Die Monster AG + Findet Nemo (Germany).

You're also right that with the -c option, it looks like the group is being broken and the other titles are no longer being associated with each other. That's definitely not desirable behavior. For future resilience when No-Intro and Redump change names, I need to ensure the remaining titles stay tied together if the key title disappears, whether it's a compilation or not. That might take a bit of tinkering, but I think it's important.

As far as compilations are concerned specifically... it's conceptually a mess. I could remove all the titles associated with those in the compilations section of the clone list (or just add them to the compilations list themselves), but I think this might have unintended consequences for different region orders/exclusions. This is because I'm currently following a standard for clone lists that likely doesn't work -- that is, only adding titles to the compilations list if their individual titles are available in the same region.

For example, these compilations are in the GBA dat:

But the individual titles are only available in the USA:

Meaning the USA compilation qualifies for the compilations list, the European one doesn't, and the European title is set as a clone of the USA version. If you exclude compilations, the USA compilation gets removed, but the European compilation remains in the dat in addition to the USA individual titles. Not ideal.

Currently I'm thinking the best path forward is to remove the option to exclude compilations, then add the individual titles as clones of the compilations with a priority of 2. This would make the compilation the selected 1G1R title in many cases. I'm quite sure there are scenarios where there's a newer version of an individual title available than what was bundled in a compilation... but for the most part I'd imagine we're probably okay. It'll need some testing, and possibly some individualized tweaking.

Taking the above example, the proposed format would look like this as a clone list entry:

"2 Games in 1 - Cartoon Network Block Party + Cartoon Network Speedway": [
    ["Double Game! - Cartoon Network Block Party & Cartoon Network Speedway", 1],
    ["Cartoon Network Block Party", 2],
    ["Cartoon Network Speedway", 2]
]

The USA compilation would then become the 1G1R title if USA is higher, the European compilation the 1G1R title if Europe is higher. This makes sense to me.

But compilations are a pain, and GBA in particular is nasty. Take this particular set of titles:

Seems okay on the surface, until you discover Finding Nemo is actually in multiple compilations, so more titles have to be considered:

Maybe with a heavily complicated set of conditional overrides it can be made to work. But gosh that's going to take a lot of thinking and testing to pull off cross-region.

For now I'll focus on the key title removal bug, then... we'll see if I can wrap my head around the more complex clone relationships.

unexpectedpanda commented 3 years ago

The bug side of this is fixed as of 0.88 -- now I just need to sort out a better way to handle compilations.

unexpectedpanda commented 3 years ago

A remove feature has been added to clone lists, which removes titles entirely from the dat instead of trying to assign them as clones. This happens whenever you run Retool, and effectively works as if the compilations filter was always on. As a result, excluding compilations is no longer a feature.

This should help whittle things down a little to get closer to the 1G1R ideal. I've managed to get around the mind explosion that was SMS compilations across multiple regions... GBA will have to wait for another day when I can put aside ample time.

ghost commented 2 years ago

If it helps, here's my most up-to-date list for GBA. Except for SN Systems (Europe), these are all compilations.

Retool version 1.01 Retool options aAbBcdDemMopPrv Clonelist last updated 22 October 2021 Datfile Nintendo - Game Boy Advance (Parent-Clone) (20211110-065833).dat

unexpectedpanda commented 1 year ago

Thanks both for your contributions and patience. After a huge engineering effort, this has finally been fixed as of v2 beta 8.