llde / xOBSE

Oblivion Script extender source
243 stars 35 forks source link

sv_Replace(StringVar::Replace function in StringVar.cpp) does not take into account when the target is a multibyte character(Problems occurred with Ultimate Leveling in Japanese environment.) #229

Open oto-git opened 1 month ago

oto-git commented 1 month ago

When Experience based leveling is selected in Ultimate Leveling, When the skill name is set to Japanese, the skill level does not increase even if the points are allocated to Mysticism.

I checked the Ultimate Leveling script, and found that when assigning skill points, there is a process that converts "_" to " " using sv_Replace in the normalization of skill names, as shown below, but at this time a misconversion occurred, and the misconverted characters were used in the subsequent process, causing the problem.

*mytile is the name of the skill

Begin Function { mytile }

set i to 1 + sv_Find "\" mytile
while i > 0
    sv_Erase mytile 0 i
    set i to 1 + sv_Find "\" mytile
loop

; REPLACE _ WITH SPACE
sv_Replace "_| " mytile

SetFunctionValue mytile

If the character in the mytile variable was "神秘", it was mis-converted to "??秘" with svReplace "| ". (For environments with Japaneseized skill names, Mysticism is named "神秘".)

I checked the StringVar::Replace function in StringVar.cpp, which handles sv_Replace, and found that it assumes that the string used is a single-byte character. It seems that the character to be searched for is treated as a single-byte character, not a multibyte character, and may be incorrectly converted if the character to be replaced in svReplace specifies a single-byte character. (In the previous example of misconversion, the character code for the search target "神秘" in bytes is 90 5F 94 E9, the character code for the character "" before substitution is 5F, and the character code for the character " " after substitution is 20, so 90 5F 94 E9 is converted to 90 20 94 E9, which is converted into the character "??秘")

llde commented 1 month ago

Thanks for the report, quite the important issue here. Most of StringVar may assume Single byte character, and I need to check how to handle this properly. The first step is a small script I can use to reproduce the problem in autonomy.