Misaligned editor cursor movement with Hangul Jamo characters

[ruby3141]

Does this issue occur when all extensions are disabled?: Yes

• VS Code Version: 1.87.2
• OS Version: Windows 11 23H2 22631.3296

[!note] (updated 2024-03-18) It turns out the majority of described problem below was not from VSCode's bug. It was VSCodeVim to blame. Sorry for misinformation. Korean with Hangul Jamo still has problem tho.

Steps to Reproduce:

• Example text

  Ｔｈｅ　ｑｕｉｃｋ　ｂｒｏｗｎ　ｆｏｘ　ｊｕｍｐｓ　ｏｖｅｒ　ａ　ｌａｚｙ　ｄｏｇ．　ｔｈｅ　ｑｕｉｃｋ　ｂｒｏｗｎ　ｆｏｘ
  The quick brown fox jumps over a lazy dog. The quick brown fox jumps over a lazy dog. The quick brown fox jumps over a lazy dog.
  とりなくこゑす / ゆめさませ / みよあけわたる / ひんがしを / そらいろはえて / おきつへに / ほふねむれゐぬ / もやのうち
  Victor jagt zwölf Boxkämpfer quer über den großen Sylter Deich Victor jagt zwölf Boxkämpfer quer über den großen Sylter Deich
  정 참판 양반댁 규수 큰 교자 타고 혼례 치른 날. 정 참판 양반댁 규수 큰 교자 타고 혼례 치른 날. 정 참판 양반댁 규수 큰 교자 타고 혼례 치른 날.
  정 참판 양반댁 규수 큰 교자 타고 혼례 치른 날. 정 참판 양반댁 규수 큰 교자 타고 혼례 치른 날. 정 참판 양반댁 규수 큰 교자 타고 혼례 치른 날.
  เป็นมนุษย์สุดประเสริฐเลิศคุณค่า กว่าบรรดาฝูงสัตว์เดรัจฉาน จงฝ่าฟันพัฒนาวิชาการ อย่าล้างผลาญฤๅเข่นฆ่าบีฑาใคร ไม่ถือโทษโกรธแช่งซัดฮึดฮัดด่า
  鉴于对人类家庭所有成员的固有尊严及其平等的和不移的权利的承认，乃是世界自由、正义与和平的基础。鉴于对人类家庭所有成员的固有尊严及其平等的
  鑑於人類社會個成員儕有個固有尊嚴脫仔平等個脫仔勿移個權利承認，是世界自由、正義脫仔和平個基礎。鑑於人類社會個成員儕有個固有尊嚴脫仔平等個

  Each line of string is consist of characters used in

  • English, with full-width latin characters
  • English, with half-width latin characters
  • Japanese
  • German
  • Korean, with Hangul Syllables Unicode block characters
  • Korean, with Hangul Jamo Unicode block characters
  • Thai
  • Mandarin, simplified
  • Mandarin, traditional

  respectively. Note that for Thai and Hangul Jamo characters, some groups of Unicode characters can be combined to compose single displayed character.

• Fonts used to create sample images NeoDunggeunmo for latin and korean glyph GNU Unifont for fallback. Both are 8x16 grid bitmap based monospace font, so glyphs are perfectly aligned. (I think this will be reproducible just with Unifont.)

• Steps

  1. Click left side of the first occuring of full-width character ａ to put cursor.
  2. Press up and down arrow key to move cursor up and down respectively.

• Expected behavior Cursor moves vertically, without horizontal movement, follwing marked line on image.

• Current behavior Cursor bumps horizontally while moves vertically, ignoring character width and combination.

[ruby3141]

Sorry for huge misinformation. VSCodeVim was not properly disabled during testing. Major fault for this problem was on VSCodeVim.

Everything works as intended EXCEPT for "Korean with Hangul Jamo", as of my reproduction.

[ruby3141]

Record of current behavior, with extensions properly disabled

You can see that the cursor clearly bumps on Hangul Jamo line.

[alexdima]

The width of the glyphs is 100% controlled by the font. Have you tried a font like Inconsolata which is known for making all glyphs equallly wide?

[ruby3141]

The glyphs of fonts I used in that reproduction are "uniformly" wide. More specifically, every "full-width" glyphs like CJK has exactly double the width of "half-width" glyph. (It's monospaced by design, but not technically. CJK programming fonts are usually behave like that.)

And VSCode simply consider full-width characters as "double sized" while calculating vertical movement. You can test that behavior with fonts like Last Resort, which is special purpose fallback font to indicate what Unicode block the "tofu"ed character is in, and has "equally" sized square box glyphs across every possible Unicode code point.

With the font, the example text looks like this. highlighted characters are in the same "VSCode vertical line". And as you can see, unlike Hangul Syllables(5th line), Hangul Jamo(6th line) seems to be considered as "half-width", by the fact that it's in the same "Displayed vertical line" with "half-width" characters like English, German and Thai.

[alexdima]

And VSCode simply consider full-width characters as "double sized" while calculating vertical movement

That is correct, that is the current implementation. With the current stack we have, we delegate font rendering in the editor to the browser, so we don't implement text layouting ourselves. To make true width based vertical movement we would need to basically render the line the cursor would land on and do an offset search to find out what would be the closest column to the current horizontal offset. The current implementation hard-codes a heuristic that wide characters are twice as wide as narrow ones. This is done at https://github.com/microsoft/vscode/blob/0a417782b1781d39803b8f1791cfb22f8835ddae/src/vs/editor/common/core/cursorColumns.ts#L37-L55 and at https://github.com/microsoft/vscode/blob/0a417782b1781d39803b8f1791cfb22f8835ddae/src/vs/editor/common/core/cursorColumns.ts#L27-L35

Because of this, the current implementation also does not work correctly for proportional fonts. I acknowledge this behavior but will mark this as a feature request because the code is working as it was built, allowing for quick vertical movement while sacrificing correctness for non-monospace fonts or mostly non-programming text content.

[VSCodeTriageBot]

This feature request is now a candidate for our backlog. The community has 60 days to upvote the issue. If it receives 20 upvotes we will move it to our backlog. If not, we will close it. To learn more about how we handle feature requests, please see our documentation.

Happy Coding!

[ruby3141]

@alexdima I don't meant about "move cursor vertically according to actual glyph size of current font" thing. It's that the Hangul Jamo block characters are not considered full width despite it is.

[alexdima]

@ruby3141 This is defined here -- https://github.com/microsoft/vscode/blob/d8abddce40aee62b166cbdc279e5c60fd9ec4fbf/src/vs/base/common/strings.ts#L677-L721 . Would you be interested in improving that with a PR?