ZW-letters that are "unmarked" should still exist

[GoogleCodeExporter]

Typing "ua" in Shan will re-order the letters (bad) due to the font itself if 
U200B is an "unmarked" whitespace character --this is because we remove it from 
the string entirely. 

We could print it as an empty space (nothing), but then the error would show up 
after typing, which is the worst time for it to manifest. 

It seems that whitespace needs to be marked, even if this might confuse users 
slightly. Perhaps we can do something like this:
  1) "whitespace-characters" identifies which letters are whitespace. 
  2) "ignored-whitespace" won't be emitted, and will be stripped from the string before drawing it too.
  3) Instead of "unmarked", we have "hide-whitespace-markings", which draws nothing (it still segments the string). 

This won't fix the problem, so then we have to remove U200B from the list of 
ignored-whitespace. The Ayar users can add this if they want to later (makes 
sense, as U200B is a flaw in Ayar, not the normal behavior). 

Should we also draw U200C/U200D? U200D should almost definitely not be handled 
by us. In that case U200C also probably shouldn't, by the same logic.

Original issue reported on code.google.com by seth.h...@gmail.com on 28 Sep 2010 at 7:30

• Blocking: #89

[GoogleCodeExporter]

There's one problem: 
   Currently, U200B still looks weird in Open Office, even for fonts that explicitly provide it (e.g., Padauk). Moreover, U200B is only needed internally by our typing systems; it really can be safely filtered except for in a few semi-non-standard cases.

   2) So, we will not strip "ignored-whitespace" from the sentence strings. 

Otherwise, our solution is good.

Original comment by seth.h...@gmail.com on 28 Sep 2010 at 7:43

• Changed state: Confirmed

[GoogleCodeExporter]

Fixed; whitespace is now much easier to draw. We won't mess up the string if 
hide-whitespace is on. 

By default, we won't show whitespace markers, but also won't output whitespace. 
This will require a section in the user's guide...

Original comment by seth.h...@gmail.com on 28 Sep 2010 at 8:50

• Changed state: Fixed