Resource resolution in [Shader/Texture]Override sections

traveling-lumine commented 2 months ago

Summary

The hash entry in the [Shader/Texture]Override sections currently only accepts a hexadecimal value (the target hash). This proposal suggests expanding the functionality to accept a namespaced resource name as an alternative to a hex value.

Proposed Feature

Enhance the hash entry to allow for an optional resource resolution if the provided value is a namespaced resource name instead of a hexadecimal hash.

Problem Statement

When the game updates resource hashes, users are required to update the corresponding hashes in their .ini files. This process is cumbersome and involves:

Manually traversing the entire mod folder structure.
Replacing old hashes with new ones, which also requires maintaining a map of old-to-new hashes.

This manual update process places a significant burden on both users and developers, leading to potential errors and inefficiencies.

Benefits

By allowing the hash entries to reference resource names, the process of updating resource hashes becomes more straightforward. Users could simply replace the .ini file with one that contains the correct resource names, eliminating the need for manual hash replacement. Additionally, using resource names is semantically clearer when reading *Override sections.

Implementation Example

This feature can be implemented by modifying the GetIniHash function in IniHandler.cpp. Below is a working example of a substitute function capable of resolving resource data while still supporting traditional hash formats.

static UINT64 GetIniHash(const wchar_t* section, const wchar_t* key, UINT64 def, bool* found)
{
    std::string val;
    UINT64 ret = def;
    int len;

    if (found)
        *found = false;

    if (!GetIniString(section, key, NULL, &val))
        return ret;

    sscanf_s(val.c_str(), "%16llx%n", &ret, &len);
    if (len != val.length()) {
        std::transform(val.begin(), val.end(), val.begin(), ::tolower);
        std::wstring_convert<std::codecvt_utf8_utf16<wchar_t>> converter;
        CustomResources::iterator res = customResources.find(converter.from_bytes(val));
        if (res == customResources.end()) {
            IniWarning("WARNING: Resource not found: %s\n", val.c_str());
            return ret;
        }
        auto& value = res->second;
        void* init_data = value.initial_data;
        if (init_data == NULL) {
            IniWarning("WARNING: initial_data unallocated");
            return ret;
        }
        size_t init_len = value.initial_data_size;
        // min is here to prevent too large data being copied
        std::string init_str((char*)init_data, min(init_len, 20));
        sscanf_s(init_str.c_str(), "%16llx%n", &ret, &len);
        if (len != init_str.length()) {
            IniWarning("WARNING: Hash parse error: %S=%s\n", key, init_str.c_str());
            return ret;
        }
    }
    if (found)
        *found = true;
    LogInfo("  %S=%016llx\n", key, ret);
    return ret;
}

Exemplary ini file contents: mod.ini:

[TextureOverrideExample]
hash = Resource\wwmi_hashes\CharacterNameAndPart
...

hashes.ini:

namespace=wwmi_hashes

[ResourceCharacterNameAndPart]
format = Buffer
data = "deadbeef"

leotorrez commented 2 months ago

This implementation doesn't completely solve the issue and has some cons to consider. It would still require a quick update of the hash list every time a new breaking version is released of the game. Which is not different from creating and maintaining a list of old to new hashes.

That is is the least of the problems however, it also makes some fixes that we currently use in other 3dm games impossible to implement in this scenario.

For example, ZZZ v1.1 remapped the weight indexes of the female main character. The easier way to track if said character has been fixed or not is to check if their ini uses the old or new hash. We can safely assume the old hash needs a hash update and a weight remap from version 1.0 to 1.1. This remap issue can happen several times across versions, like is the case of ningguang in genshin impact. Her character had I believe 3 maps already- In order to fix an old mod to the current version we need the hashes to know which remap corresponds to the mod in question.

Other similar case is format changes in buffers. Recently the head of most female characters in ZZZ got their format changed and therefore their stride as well (from 96 to 104). Granted the stride can be used as reference in this case up to a point and only when the change has happened only once, but a hash is a more certain indicator of change and versions. Due to how some mods got exported the stride was lower than 96(removing some UV maps) and could technically be higher as well(adding unnecesary UV maps) whilst still working normally- so stride is a weak indicator

SpectrumQT commented 1 month ago

After careful consideration, I've decided to reject this proposal. As leotorrez mentioned, implementing this feature would make automatic update mechanism way less reliable without introduction of the new metadata format that we can't expect all modders to abide. So, introduction of such middleman wouldn't be able to solve the update problem in general, would require update apps to take into account extra format and would only complicate automatic updates of mods using it.

SpectrumQT / WWMI