Words from some languages getting past the filter

Gasegamer / super-profanity

An advanced profanity filter based on phonetics and normal detection that detects swears in 27 different languages!

Mozilla Public License 2.0

16 stars 2 forks source link

Words from some languages getting past the filter #1

Closed Bytedefined closed 2 years ago

Bytedefined commented 2 years ago

Languages such as: Russian and French seem to be getting past the filter, but languages such as Chinese and English seem to be working just fine.

Bytedefined commented 2 years ago

I looked into the code, and to me it seems like it only checks one language, which is your main language? Are you able to make it check all languages?

Bytedefined commented 2 years ago

accidentally closed lol

Bytedefined commented 2 years ago

There also seem to be false flags for words such as "grass"

Gasegamer commented 2 years ago

Languages such as: Russian and French seem to be getting past the filter, but languages such as Chinese and English seem to be working just fine.

Hi, Firstly, thanks for the report, I will take a look at this issue with Russian and French, but if you want to have better precision on them, try setting them as the main language.

Gasegamer commented 2 years ago

I looked into the code, and to me it seems like it only checks one language, which is your main language? Are you able to make it check all languages?

Your main language serves as a guarantee that the code will check for swears in a desired language. This is necessary, because the detection of the language (which is done by the TinyLD package) is not completely accurate, especially in very short sentences.

If you look again at the code, you will see that it uses the detectAll function of that package, which detects the different possibilities of the language in which the sentence was written. However, I will check if I used it properly to make sure it is using in all possible languages.

Gasegamer commented 2 years ago

There also seem to be false flags for words such as "grass"

I'll take a look at that too and update Read-ME to show the whitelist and blacklist functions in addition to a brief explanation about main language.

Again, thanks for the report.

Bytedefined commented 2 years ago

Thanks

Bytedefined commented 2 years ago

any updates?

Gasegamer commented 2 years ago

any updates?

Currently no, sorry. I've been traveling for a few days and that's why I haven't been able to do anything for now.

Gasegamer commented 2 years ago

any updates?

Fixed the problem of some words being detected wrongly.
Fixed the checker to detect and ignore whitelisted words.
Updated the README to clarify some functionality.

Unfortunatelly, no success with the problem of bad detection in Russian and French. However, you can, for now, use the changeMainLanguage, whiteListWord and/or blackListWord functions to solve this problem.

Gasegamer commented 2 years ago

This issue was fixed in Release 1.1.1.