Detect duplicate mods

[Wolfmyths]

There should be a way to detect if there are conflicting mods in mod_overrides.

If there are conflicting mods, ask the user which one to keep enabled.

The user will have the option to ignore this operation, but the mod will be red in the UI.

[H4ppy-04]

Just a quick question, would this be checking if there are duplicate mods / folders?

[Wolfmyths]

Just a quick question, would this be checking if there are duplicate mods / folders?

No, but that should probably also be a feature.

I'm not sure if detecting conflicting assets is a good idea because I'm pretty sure there's no way to set up a mod load order without rewriting how the program sets up the mods.

Even then, I don't think it would be worth it since the mod manager doesn't inject the mods into the game, I wanna say Super BLT does that instead, but for asset replacements I'm not 100% sure.

The mod manager is just a file manager.

[costantino2000]

A simple duplicate check might be enough to see if two mods change the same file. Just having a popup with a list of conflicting mods (showing which files are duplicates) would be enough, instead of relying on an external tool.

[H4ppy-04]

Something like:

import os

def has_duplicates(path, recursive=False) -> list:
    """ Return a list of duplicate files

    path: The path to scan the files for
    recursive: Should the scan be recursively iterating through files?

    Return: a list of duplicate folders. The list is empty if there
        are no duplicates.
    """

    files = [i for i in os.listdir(path) if os.path.isfile(i)]
    duplicates = []
    for i in files:
        files.pop(i)
        if i in files:
            duplicates.append(i)

On 23/10/28 12:46PM, costantino2000 wrote: A simple duplicate check might be enough to see if two mods change the same file. Just having a popup with a list of conflicting mods (showing which files are duplicates) would be enough, instead of relying on an external tool.

-- Reply to this email directly or view it on GitHub: https://github.com/Wolfmyths/Myth-Mod-Manager/issues/17#issuecomment-1783907535 You are receiving this because you commented.

Message ID: @.***>

[costantino2000]

Sadly I discovered that it's not that easy. After using Dupeguru to try to do the same thing I saw that many mods keep the same file in different directories, and 90% of the results I found were fake duplicates. I guess a correct approach would also need to check the subpath and maybe the file size, but I don't know if it would work for every case.

[Wolfmyths]

I guess a correct approach would also need to check the subpath and maybe the file size, but I don't know if it would work for every case.

This might be too much, but do you think we could look at the actual files it's replacing and mark them as being in-use in a list for the whole function. We can use os.walk to achieve this I think. Although it may be performance heavy

[H4ppy-04]

One way to increase the performance might be to cache the files.

Maybe something like recursively iterating through the mods and then dumping the file name and file size and any other data into a .json file.

Then, when files need to be compared, the cache can be used instead of querying the actual file itself?

On 23/10/29 09:48AM, Wolfmyths wrote:

I guess a correct approach would also need to check the subpath and maybe the file size, but I don't know if it would work for every case.

This might be too much, but do you think we could look at the actual files it's replacing and mark them as being in-use in a list for the whole function. We can use os.walk to achieve this I think. Although it may be performance heavy

-- Reply to this email directly or view it on GitHub: https://github.com/Wolfmyths/Myth-Mod-Manager/issues/17#issuecomment-1784164741 You are receiving this because you commented.

Message ID: @.***>

[costantino2000]

do you think we could look at the actual files it's replacing and mark them as being in-use in a list for the whole function. We can use os.walk to achieve this I think. Although it may be performance heavy

It might work and that was my original idea, but I saw that some mods like custom outfits share the same folder structure, so they will show as duplicates even if they are not. At this point I think it should be implemented in another tab and show a warning about the possibility of fake duplicates, I don't see any other way of solving the problem easily since I'm not a modder.