Bad-words filter only works for roman character

[frdmn]

Description:

The bad-words filter, only seems to respect roman characters.

I could imagine this is because the default regex is used in the library:

https://github.com/web-mech/badwords/blob/e15b99e41ebf794503148b80699914ebcf12f173/lib/badwords.js#L21

There is a mention of multi-lingual support with an example in the project README.md:

var filter = new Filter({ replaceRegex:  /[A-Za-z0-9가-힣_]/g }); 
//multilingual support for word filtering

Steps to reproduce:

1. Go to admin UI → Message

2. Now set ...

   • "Allow Message bad words filtering" to True
   • "Add Bad Words to the Blacklist" to test,example,Привет

3. Save configuration

4. Go to any channel and send a message with the content: test,example,Привет

Expected behavior:

All of the configured "bad words" should be censored:

frdmn: ***,*******,******

Actual behavior:

frdmn: ***,*******,Привет

Server Setup Information:

• Version of Rocket.Chat Server: 0.70.4
• Operating System: Cloud
• Deployment Method: Cloud
• Number of Running Instances: Cloud
• DB Replicaset Oplog: Cloud
• NodeJS Version: Cloud
• MongoDB Version: Cloud

Additional context

[engelgabriel]

Now that we have the Content Filter app for that, we should remove the functionality from the main code altogether.

We must make sure the Content Filter app can work with the non-roman characters.