monk-afk / filterplus

Chat filter and censor for Minetest. Includes mute command, player tags, and mention highlight.
MIT License
1 stars 0 forks source link

Non-latin characters #5

Closed monk-afk closed 8 months ago

monk-afk commented 9 months ago

Only detects Latin characters.

monk-afk commented 8 months ago

possibly a locale issue,

:~$ locale -a
C
C.UTF-8
en_US.utf8
POSIX

link for future reference: https://serverfault.com/questions/54591/how-to-install-change-locale-on-debian

monk-afk commented 8 months ago

using /blacklist insert converts to utf with escape: \"\u0441\u043f\u0430\u0441\u0438\u0431\u043e\"

the word спасибо is not detected from chat

modifying the mod_storage entry from escaped string to the exact word, it is automatically converted from спасибо to \"\u0441\u043f\u0430\u0441\u0438\u0431\u043e\" during serialization

also tried with letters such as ç with no difference

monk-afk commented 8 months ago

adding to init.lua:

if word == "спасибо" then
    print("detected спасибо")
end

is successful:

detected спасибо
2024-02-04 01:41:24: ACTION[Server]: monk issued command: спасибо
monk-afk commented 8 months ago

https://github.com/MultiCraft/MultiCraft/blob/fc02a37cdd2114842fab55b70050c5b98cdf9f11/doc/lua_api.txt#L6964

https://github.com/minetest/minetest/blob/e7dbd325d2e50e7bc3a509ec6a7eaeee1ec2ae8a/doc/lua_api.md?plain=1#L8350

monk-afk commented 8 months ago

changing line 147 to character class set works on minetestserver 5.9.0-dev, as well as multicraftserver 2.0.6

local word = gsub(lower(try_word), "[%p%c]", "")

image

./minetestserver --version
Minetest 5.9.0-dev (Linux)
Using LuaJIT 2.1.0-beta3
Running on Linux/6.1.0-9-amd64 x86_64
BUILD_TYPE=Release
 ./multicraftserver --version
MultiCraft 2.0.6 (Linux)
Using LuaJIT 2.1.0-beta3
BUILD_TYPE=Release