Some characters in Unicode have the property Default_Ignorable_Code_Point. This includes characters like the ZWNJ zero width non-joiner U+200C. Most format characters have this property. With CTL and OpenType support available in Scribus, the usage of these characters will increase.
These characters have yet a special treatment in spell checking since #215. The question is how to treat these characters in hyphenation. Currently, the automatic hyphenation does not work as expected when characters like ZWNJ are present. Example: The German word “Auflage” should be hyphenated “Auf-la-ge”, and this works correctly in Scribus when no ZWNJ is there. But when there is a ZWNJ between “Auf” and “lage” then the first hyphenation point is not found. It’s only “Aufla-ge”.
Possible solution:
Do nothing. (At least the current situation does not create much harm.)
Support Default_Ignorable_Code_Point characters as expected. Might have more overhead (mapping of character positions) but might be the best solution. At scribus/text/specialchars.cpp there is bool specialChars::isIgnorableCodePoint(uint ch) that might be usefull.
sommerluk
commented
7 years ago
Some characters in Unicode have the property Default_Ignorable_Code_Point. This includes characters like the ZWNJ zero width non-joiner U+200C. Most format characters have this property. With CTL and OpenType support available in Scribus, the usage of these characters will increase.
These characters have yet a special treatment in spell checking since #215. The question is how to treat these characters in hyphenation. Currently, the automatic hyphenation does not work as expected when characters like ZWNJ are present. Example: The German word “Auflage” should be hyphenated “Auf-la-ge”, and this works correctly in Scribus when no ZWNJ is there. But when there is a ZWNJ between “Auf” and “lage” then the first hyphenation point is not found. It’s only “Aufla-ge”.
Possible solution:
scribus/text/specialchars.cppthere isbool specialChars::isIgnorableCodePoint(uint ch)that might be usefull.