[BUG - Release] mods.txt parsing

[narknon]

Branch or Release Release and main

Game and Engine Version Any

Describe the bug First line of mods.txt is read as having additional characters.

To Reproduce Print the first mod's name to console or otherwise.

Expected behavior No extra characters

Screenshots, UE4SS Log, and .dmp file

[UE4SS]

Note: This is regarding mods.txt, not mods.json that's showed in the screenshot.

[narknon]

Can be fixed by simply recreating our mods.txt, but this problem is likely pervasive since last release with anyone's files. Unclear when it became corrupted without more testing. So, it should probably be fixed programmatically if at all possible.

[UE4SS]

Duplicate of #486

[narknon]

Corrupted mods.txt was introduced between 2.5.2 and 3.

[narknon]

Corrupted mods.txt was not introduced as part of a commit. Possibly related to our release action's method of copying instead.

https://github.com/UE4SS-RE/RE-UE4SS/commit/63ce8b427c7afca8e09f501035fd706eb0198104 - does not have the issue.

[narknon]

Solution would be to skip  chars / EF BB BF hex when reading mods.txt for the foreseeable future.

[narknon]

Something like this should probably go into our File module to use when opening file streams. I don't feel like doing that at the moment though.

    // BOM sequences
    const std::string BOM_UTF8 = "\xEF\xBB\xBF";
    const std::wstring BOM_UTF16_LE = L"\xFEFF";
    const std::wstring BOM_UTF16_BE = L"\xFFFE";

    // Function to check for BOM and reopen the file as a wide stream
    std::wifstream openFileSkippingBOM(const std::wstring& filename) {
        std::ifstream file(filename, std::ios::binary);
        if (!file.is_open()) {
            throw std::runtime_error("Failed to open the file.");
        }

        char bom[3] = {0};
        file.read(bom, 3);
        if (std::string(bom, 3) == BOM_UTF8) {
            // UTF-8 BOM detected
            file.close();
            std::wifstream wfile(filename);
            wfile.seekg(3, std::ios::beg); // Skip BOM
            return wfile;
        } else {
            // Check for wide BOM
            wchar_t wbom;
            file.seekg(0, std::ios::beg); // Rewind to start
            file.read(reinterpret_cast<char*>(&wbom), sizeof(wchar_t));
            file.close();
            if (wbom == BOM_UTF16_LE[0] || wbom == BOM_UTF16_BE[0]) {
                std::wifstream wfile(filename);
                wfile.imbue(std::locale(wfile.getloc(), new std::codecvt_utf8_utf16<wchar_t>));
                wfile.seekg(sizeof(wchar_t), std::ios::beg); // Skip BOM
                return wfile;
            } else {
                // No BOM detected
                file.close();
                return std::wifstream(filename);
            }
        }
    }

[narknon]

Fixed by #556 and #540