computerline1z / okapi

Automatically exported from code.google.com/p/okapi
0 stars 0 forks source link

Checkmate ”Blacklist” function does not work for Japanese #442

Closed GoogleCodeExporter closed 9 years ago

GoogleCodeExporter commented 9 years ago
I tried "Blacklist" function for English to Japanese translation.
But it does not work at all.

What is the expected output? What do you see instead?

Probably, I guess that "Blacklist" works as if "¥b" is put before and after a 
listed word. 
For example, if "function" is a blacklisted word, Checkmate responds to 
"function", but not "malfunction".
This is convenient for European language, but it is critical for Japanese 
(probably, Chinese, Korean, too).
Because, in most cases, Asian language does not contain any "space" in a 
sentence.

What version of the product are you using? On what operating system?
Checkmate version 0.26

I hope your support.

Original issue reported on code.google.com by tsuruku...@gmail.com on 4 Feb 2015 at 1:39

GoogleCodeExporter commented 9 years ago
The blacklist checker doesn't use regex like \b, but does checks for character 
types. So the algorythm needs to be updated to work with Japanese, etc.

Could you provide an example oj Japanese of a translated string and several 
black listed terms, so we can try to tweak the code? Thanks. 

Original comment by yves.sav...@gmail.com on 4 Feb 2015 at 2:00

GoogleCodeExporter commented 9 years ago
Thank you for your quick reply.
I created simple file attached.
One is Trados.ttx, another is Blacklist UTF-8.

Original comment by tsuruku...@gmail.com on 5 Feb 2015 at 1:51

Attachments:

GoogleCodeExporter commented 9 years ago
Is my request hard to follow?
If you have any questions or need more sample, I will support you.

Original comment by tsuruku...@gmail.com on 12 Feb 2015 at 2:34

GoogleCodeExporter commented 9 years ago
Hi there, Yves is out of the office until next week but I'm planning to take a 
look at it tomorrow.

Original comment by fli...@enlaso.com on 13 Feb 2015 at 12:37

GoogleCodeExporter commented 9 years ago
Ok, this snapshot has an option to allow match blacklist terms even if they are 
substrings. 

https://okapi.ci.cloudbees.com/job/okapi-snapshot/

Original comment by fli...@enlaso.com on 14 Feb 2015 at 3:02

GoogleCodeExporter commented 9 years ago

Original comment by yves.sav...@gmail.com on 19 Feb 2015 at 3:31