Feature request: Option to reduce compilation duplicates even more

[tpmoney]

In some systems games were released in both individual forms and multi pack games. For example the GBA had a number of "2 in 1" packs, or the NES Super Mario + Duck Hunt cartridge. Right now the 1G1R filtering prefers to drop the individual games for their X in Y collection and while it's possible to build up a set of partial matches / regexes to drop them, it isn't quite as simple as just matching against "2 in 1" because some games / cartridges were only released as a combo pack. It would be handy to have a option to say that you prefer the individual games over their collections when all the parts of a collection were released as individual games.

[unexpectedpanda]

As of v2 beta 8 individual games are preferred by default, unless the equivalent compilation supports a higher language, features a higher primary region, or is set higher in the clone list. You might be running an old version -- the current version of Retool as of writing has exited beta and is the final 2.00.0.

[tpmoney]

That would be it, I was running beta 7. Re-ran against the list and my hand filtering and found this difference:

TITLES WITH CLONES
==================
...
+ Classic NES Series - Dr. Mario (USA, Europe)
  - 2 Games in 1 - Dr. Mario + Puzzle League (Europe) (En,Fr,De,Es,It)
  - Dr. Mario & Panel de Pon (Japan)
  - Famicom Mini 15 - Dr. Mario (Japan)

...

TITLES WITHOUT CLONES
==================

+ 2 Games in One! - Dr. Mario + Puzzle League (USA)

From a trace it looks like this is because Puzzle League wasn't ever released outside of the combo pack. Might be worth having an option that removes collections if and only if all the games in the collection are available as individual games.

Appreciate it and all the work you put into this tool.

[unexpectedpanda]

I might be misunderstanding, but that's exactly how it works now. Compilations get kept if they contain a title not available individually.

The basic approach is they are split into virtual versions of their constituent titles before any comparison is done. If that virtual title is the only one in the group, then the associated compilation ends up being kept even though one of its other titles might already be represented as an individual title.

It does occasionally make what looks like a confusing clone assignment at first glance, but that's the nature of the beast. The DAT format only allows a single parent to be defined, so that's what I've followed. The final titles selected should still be correct.

In your example, it ultimately kept both Dr. Mario and Puzzle League USA versions. You got the standalone title Classic NES Series - Dr. Mario (USA, Europe) for Dr. Mario, and the compilation 2 Games in One! - Dr. Mario + Puzzle League (USA) for Puzzle League.

In previous versions of Retool only the compilation would have been kept for the sake of saving space, but there's a strong preference in the user base for keeping individual titles where possible for patches, retro achievements, and knowing what titles you have on your hard drive as some compilation names don't give away what games they contain.

[tpmoney]

Yeah I didn't explain the option suggestion well enough. In the case of the example I was suggesting an option that says

Because Puzzle League was never released standalone, then prefer the compilation and don't include Classic NES Series - Dr. Mario

If Puzzle League had been available as a standalone, prefer the individual games. The default settings make sense, and I can certainly see where coding that logic would be a lot of complexity for a not very common concern that for now could be addressed with user filters.

As it stands the default behavior is more than adequate for my preferences, so if that option isn't something you think anyone would really benefit from, I'm happy to consider this issue closed.

[unexpectedpanda]

Got it, you're looking to reduce duplicates further where possible. That was originally in scope as an option, but compilations are complex so I opted for a single path after early experiments didn't work out.

I'll leave this as a feature request for now, maybe one day.

[prenetic]

I might be misunderstanding, but that's exactly how it works now. Compilations get kept if they contain a title not available individually.

In previous versions of Retool only the compilation would have been kept for the sake of saving space, but there's a strong preference in the user base for keeping individual titles where possible for patches, retro achievements, and knowing what titles you have on your hard drive as some compilation names don't give away what games they contain.

Not to hijack this thread but this is something I just noticed and was digging around trying to understand the reasoning. Is Super Mario All-Stars not considered a compilation, or is this an error in the clone list? There is a standalone USA release of Super Mario World, yet it's discarded in favor of the compilation using default settings with the latest version Retool and clone lists.

I strongly agree with the preference for standalone titles for precisely the reasons listed, but it doesn't appear that's what's happening here. I can only surmise this is happening because they are considered part of the same superset, however that seems orthogonal to the stated goal above.

No-Intro P/C DAT

    <game name="Super Mario All-Stars + Super Mario World (USA)" cloneof="Super Mario All-Stars and Super Mario World (Europe)">
        <description>Super Mario All-Stars + Super Mario World (USA)</description>
        <release name="Super Mario All-Stars + Super Mario World (USA)" region="USA"/>
        <rom name="Super Mario All-Stars + Super Mario World (USA).sfc" size="2621440" crc="f84305b1" md5="7ecaf3e2e021c8a836041c85886f4594" sha1="d245e41a2b590f7d63666b0772cbddfb26f254a2" status="verified"/>
    </game>

    <game name="Super Mario World (USA)" cloneof="Super Mario World (Europe) (Rev 1)">
        <description>Super Mario World (USA)</description>
        <release name="Super Mario World (USA)" region="USA"/>
        <rom name="Super Mario World (USA).sfc" size="524288" crc="b19ed489" md5="cdd3c8c37322978ca8669b34bc89c804" sha1="6b47bb75d16514b6a476aa0c73a683a2a4c18765" status="verified"/>
    </game>

Clonelist L2365-2375

        {
            "group": "Super Mario All-Stars",
            "titles": [
                {"searchTerm": "Super Mario All-Stars"},
                {"searchTerm": "Super Mario Collection"}
            ],
            "supersets": [
                {"searchTerm": "Super Mario All-Stars + Super Mario World"},
                {"searchTerm": "Super Mario All-Stars and Super Mario World"}
            ]
        },

L2383-2393

        {
            "group": "Super Mario World",
            "titles": [
                {"searchTerm": "Super Mario World"},
                {"searchTerm": "Super Mario World - Super Mario Bros. 4"}
            ],
            "supersets": [
                {"searchTerm": "Super Mario All-Stars + Super Mario World"},
                {"searchTerm": "Super Mario All-Stars and Super Mario World"}
            ]
        },

[unexpectedpanda]

The compilation version of Super Mario World has a few tweaks compared to the standalone, making it a superset. It's not an untouched collection of the two titles.

[rufotheone-fr]

Hi ! Could it be possible to have a tickable option to "prefer compilations over single title" ?

[unexpectedpanda]

I think I've tracked five or six different ways people would like compilations processed. I definitely understand that people want to organize things in a way that fits their specific scenario.

To set realistic expectations, all things compilations are very low priority for me. Getting it to where it is now took considerable effort, and I'm loathe to add more complexity to that part of Retool.

That's not saying never; just that many other things are likely to be completed before I come back to this.

[unexpectedpanda]

Compilation handling options have been added in v2.03.0.