Better way for ParallaxGen to identify textures across incompatible mods

[hakasapl]

Current use cases that are problematic

1. Overwrites in loose files that result in a diffuse/normal not matching with env mask/height map
   1. For example, adding a non-parallax and non-CM retexture after an AIO that has parallax or CM, ParallaxGen will use the height map / env mask of the AIO in combination with the new diffuse, resulting in issues.
2. Inferior shaders overwriting superior ones
   1. For example, a user might install a parallax mod that has height maps. If a matching CM env mask existing higher up in the loose file load order, ParallaxGen will use that instead, resulting in issues.
3. Wanting to use an env mask from --upgrade-shaders instead of the env mask already in the load order
   1. For example if we wanted to use a parallax mod with CM, currently --upgrade-shaders won't generate an env mask because an existing env mask already exists.

The core issue is that ParallaxGen cannot identify whether a diffuse/normal map goes with a height/env mask map

Potential Solutions

1. Use image processing techniques to determine if diffuse/normal goes with height/env mask
   1. This was attempted unsuccessfully using edge detection techniques, among others. For some things it worked, but there was a lot of false results
2. Use an ML model to determine if they match
   1. Requires a large dataset of maps that both go together and that do not go together. This would require using modder's assets to train the dataset - use permissions?
   2. Will never be 100% accurate - the nature of ML
3. Hooking mod managers
   1. Will require support for MO2 and vortex, and it would have to be optional, some people still use wyre bash etc.

Mod Manager Solution

1. Figure out a way we can map a specific texture to the loose file load order (so we can see if two files are above or below on the loose file load order from each other)
2. Use that info to decide the shader. The winning texture becomes the shader (ie. parallax could win over CM if it's further down in the loose file load order)
3. In the case of --upgrade-shaders, if a parallax map is higher priority than a CM map, it should still make a new CM map with that parallax map instead of skipping.

@Sebastian-81 thoughts?

[Sebastian-81]

I also see the Mod manager solution as the most reliable solution and we could make it optional in case someone does not want to use a mod manager. Also for that case and as a first step in general we could:

• Output a summary of parallax and cm material maps found and their corresponding diffuse maps and their origin, BSA or loose. This could be part of a larger summary project which I think could be very useful in general. DynDoLod uses summaries and very detailed descriptions which I find help a lot
• Use the mod manager extension mechanisms to at least warn the user if
  • a parallax map is installed without a diffuse map
  • a diffuse map is installed and an existing parallax map of the same name is already present

[Sebastian-81]

I just had another idea: The report could show small previews of the used height map, cm map and their corresponding diffuse texture. Then it would be very easy to spot if there is a mismatch.

[Sebastian-81]

I just realized that we can find files in the staging folder of the mod manager, for Vortex on my machine it isC:\ProgramData\vortex\skyrimse\mods - we can do a mapping of the textures to the mod directory and in the summary output a warning when the diffuse texture is from a different directory than the parallax texture. We could match the texture by filesize.

For MO2 I don't know yet, what do you think @hakasapl ?

[Sebastian-81]

As discussed: MO2 has the same folder, so we can make use of that folder information. For MO2 there is also a load order file that can be used

Use case "find out if parallax and diffuse don't match"

If the user specifies a command line argument with the MOD manager staging folder, we create a summary file that shows mod names if one mod contains the diffuse and the other the parallax map based on the load order. This is ensured by comparing the actual file used in the data folder via hash and filename to the one of the staging folder. The folder where this texture lies is the winning mod.

• pre-fill file map from directories with hash and filename
• ouput file with summary of mismatches
• the config file has a list of mod pairs that are allowed, i.e. where the parallax was added after the original mod was created