Improvement handling of multibyte characters

[berlysia]

When we type multibyte chalacters, as follows:

$ ruby repl.rb
>> "abcdef"
=> "abcdef"
>> "あ                                                                                                                                       >> "あい                                                                                                                                     >> "あいう                                                                                                                                   >> "あいうえ                                                                                                                                 >> "あいうえお                                                                                                                               >> "あいうえお"
=> "あいうえお"
>>

This PR makes like this:

$ ruby repl.rb
>> "abcdef"
=> "abcdef"
>> "あいうえお"
=> "あいうえお"
>>

[Mon-Ouie]

Hey,

Thanks, I had noticed that bug but didn't understand why it was happening.

Now the problem with your approach is that only a few unicode characters (or character classes) have a length that isn't 1, so this breaks strings like "àïëöüùπΣ". I'd have to see what's the most convenient way to get a character's expected width.

[berlysia]

I'm sorry for my ad-hoc code. Thanks for your reply.

UAX #11: East Asian Width, It seems that key to this problem.
This is already implemented as a gem, janlelis/unicode-display_width.

"あいうえお".display_width => 10
"àïëöüùπΣ".display_width => 8

Looks good? More test is needed, but this looks effective.

[Mon-Ouie]

Hey,

I used another gem (which also has a few more methods I may want to use in the future) in f5d3d5f, does that work for you? Specifically, I'm not sure it works correctly for lines that are too long to fit and require horizontal scrolling.

[berlysia]

It works very good! Actual and apparent cursor positions are conflicted in my commit, but I can't find where caused (it's terminal's problem I recognised).

Recoding some cursor behavior when required horizontal scrolling.

It works as expected as my typing. pretty good. At least, there are no inconvenience when typing Japanese words.

[Mon-Ouie]

The new gem fixed the issue ;)

I think the reason the cursor was displayed at the wrong column is simply because Cooline keeps track of the position using string indices, not columns. I kept it that way because it's easier to do string manipulation that way, but when printing, indices need to converted to column position for it to work properly (by calculating the width of string[0...i]).