Soft wrap for CJK

[raccy]

Fix: atom/atom#1783.

Summary

• Use measureText(), so you can use proportional font.
• Use Automattic/node-canvas instead of document.createElement("canvas"). But chromium can use document... maybe, so can remove node-canvas in Atom etc...
• Specify font-size, font-family, and the base character. Specs use 'Georgia', 'Courier New', and 'Noto Sans (with CJK)' fonts.
• Selectable simple soft wrap or line breaking rule soft wrap with a proportional font.
• Line breaking rule refer to wikipedia:Line breaking rules in East Asian languages.

  Please fix me!

• Need to add args new SoftWrapsTransform(...) in TextDisplayDocument with font-size and font-family.
• Add Chinese and Korean texts to the specs.
• Poor performance (-д-;

[saschanaz]

:+1: but nobody responds... This bug is so major to me but it seems Atom team don't think so.

[borrrden]

I support you both! I cannot use Atom unless this issue is resolved.

[nathansobo]

Hi there. Thanks for doing all this work and investigation. Apologies for the delayed response. We've been pretty focused on a few big things for 1.0 and have let things get backed up a bit. It looks like a lot of careful thought went into this.

Unfortunately, we've decided not to continue forward with text-document because it was too big of a change. Most of the work has been folded into Atom's existing editor implementation, which isn't as pretty but is also integrated and already working, which has its advantages. Probably should have stuck with iterating on that to begin with, but we needed to explore. We would however be very interested in integrating what you've learned on this repo in atom/atom itself.

It seems like this PR should really be broken into two different steps. First, we need to update our approach to character measurement in Atom in general, to be something much more along the lines of what you're implementing. As you can imagine, we don't want to just measure characters for the purposes of soft wrapping. We use these measurements for positioning the cursor and other elements of the UI, so we should integrate soft wrap with that.

In atom/atom#6083 there's been some progress on updating measurement, but so far we've found a canvas based approach to be too slow. I will write a better explanation of the situation in that PR if you're still interested in pushing this forward.

Once our measurement issues are fixed, the DisplayBuffer of the current implementation in atom/atom is where we'd need to apply this change. Just a warning: We have some pretty dramatic changes planned at some point for that layer to clean up the representation and make it more compact, but we should be able to coordinate with you.

[saschanaz]

@nathansobo Does it mean I can reopen https://github.com/atom/atom/pull/6425?

[nathansobo]

Yes. Sorry for the runaround. Appreciate your patience as we figure things out.

[nathansobo]

I'll comment on that PR when I get back to my computer.

[raccy]

I see. I'm waiting to be fixed in a better way. (If none, I'll be troubled, because I have wrote that my soft-wrap package will be needed in Atom 1.0...(TдT)) I have a little skill, but I'll help you as much as possible.

[nathansobo]

@raccy Would there be any value in just soft-wrapping at N characters but actually counting those characters correctly? Do CJK characters vary in width so dramatically that this wouldn't be useful? Then we could apply a solution for variable-width soft wrap in a subsequent step.

[raccy]

@nathansobo When we use monospace font (MS ゴシック, Osaka, etc.), YES. Almost Japanese programers write code with monospace font. Reason:

1. With some traditional multibyte encoding (Shift_JIS, EUC-JP), the string width equals the number of bytes that needed to represent it.
2. Japanese printed matter (news papers, books, etc.) use monospace font, so we are familiar it.

So my package (japanes-wrap) counts just x0/x1/x2 width of each characters, it is enough for me. But, please note that some characters (Greek/Coptic and Cyrillic) are different width depending on the font.

Sorry I don't know well Chinese and Korean. Also please see and ask the same question in atom/atom#1738.

[bgever]

I've noticed that Visual Studio Code, which is based on Atom but uses a different editor, doesn't have this problem and has soft-wrap as expected. Hopefully they open-source it soon, so that their implementation could be understood.