For a character like 者, which is a 3-byte UTF-8 character, with bytes (232, 128, 133), it would get expanded by the replace_termcodes call to (232, 128, 254, 88, 133). This is due to special handling around the 128 byte - it gets expanded via K_SPECIAL to its own 3-byte sequences.
This only affected a subset of UTF-8 characters - ones without a 0x80/128 byte would not be impacted by this replacement.
This breaks my assumption that replace_termcodes is safe for arbitrary Unicode values - so I split the vimInput API into two APIs:
vimInput - pass the characters w/o replace_termcodes. Safe for Unicode, but does not handle keys like <LEFT>, <ESC>, <CR>, etc.
vimKey - pass the characters into replace_termcodes. Not safe for Unicode, but handles keys like <LEFT>, etc.
This is the
libvim-side fix for https://github.com/onivim/oni2/issues/1720For a character like
者, which is a 3-byte UTF-8 character, with bytes (232, 128, 133), it would get expanded by thereplace_termcodescall to (232, 128, 254, 88, 133). This is due to special handling around the128byte - it gets expanded viaK_SPECIALto its own 3-byte sequences.This only affected a subset of UTF-8 characters - ones without a
0x80/128byte would not be impacted by this replacement.This breaks my assumption that
replace_termcodesis safe for arbitrary Unicode values - so I split thevimInputAPI into two APIs:vimInput- pass the characters w/oreplace_termcodes. Safe for Unicode, but does not handle keys like<LEFT>,<ESC>,<CR>, etc.vimKey- pass the characters intoreplace_termcodes. Not safe for Unicode, but handles keys like<LEFT>, etc.