A frontend-independent IDE "smartness" server for Elixir. Implements the JSON-based "Language Server Protocol" standard and provides debugger support via VS Code's debugger protocol.
Apache License 2.0
846
stars
52
forks
source link
Formatter should use UTF-16 string representation #215
There is a mismatch during the handoff of what character positions to change from elixir-ls to VSCode and vice versa (and I assume all other editors).
Some intuition:
In JS: '😀'.length === 2 (because strings are represented in UTF-16)
In Elixir: String.length("😀") === 1 (because strings are represented in UTF-8)
Format file (default Alt+Shift+F in VSCode or equivalent)
It will format into this (notice the parenthesis placement):
IO.inspect("😀)"
How it should format:
IO.inspect("😀")
IO.puts "🏳️🌈" turns into IO.puts("🏳)️🌈" as 🏳️🌈 is length 6 in UTF-16; it's splitting the grapheme.
Similar behavior happens for Zalgo strings like so: ẕ̸͇̞̲͇͕̹̙̄͆̇͂̏̊͒̒̈́́̕͘͠͝à̵̢̛̟̞͚̟͖̻̹̮̘͚̻͍̇͂̂̅́̎̉͗́́̃̒l̴̻̳͉̖̗͖̰̠̗̃̈́̓̓̍̅͝͝͝g̷̢͚̠̜̿̊́̋͗̔ȍ̶̹̙̅̽̌̒͌͋̓̈́͑̏͑͊͛͘ ̸̨͙̦̫̪͓̠̺̫̖͙̫̏͂̒̽́̿̂̊́͂͋͜͠͝͝ṭ̴̜͎̮͉̙͍͔̜̾͋͒̓̏̉̄͘͠͝ͅę̷̡̭̹̰̺̩̠͓͌̃̕͜͝ͅͅx̵̧͍̦͈͍̝͖͙̘͎̥͕̾̾̍̀̿̔̄̑̈͝t̸̛͇̀̕
There is a mismatch during the handoff of what character positions to change from elixir-ls to VSCode and vice versa (and I assume all other editors).
Some intuition: In JS:
'😀'.length === 2
(because strings are represented in UTF-16) In Elixir:String.length("😀") === 1
(because strings are represented in UTF-8)LSP text documents are in UTF-16 string representation: https://microsoft.github.io/language-server-protocol/specification#textDocuments
Repro:
Paste this into a blank Elixir file:
Format file (default Alt+Shift+F in VSCode or equivalent) It will format into this (notice the parenthesis placement):
How it should format:
IO.puts "🏳️🌈"
turns intoIO.puts("🏳)️🌈"
as🏳️🌈
is length 6 in UTF-16; it's splitting the grapheme.Similar behavior happens for Zalgo strings like so: ẕ̸͇̞̲͇͕̹̙̄͆̇͂̏̊͒̒̈́́̕͘͠͝à̵̢̛̟̞͚̟͖̻̹̮̘͚̻͍̇͂̂̅́̎̉͗́́̃̒l̴̻̳͉̖̗͖̰̠̗̃̈́̓̓̍̅͝͝͝g̷̢͚̠̜̿̊́̋͗̔ȍ̶̹̙̅̽̌̒͌͋̓̈́͑̏͑͊͛͘ ̸̨͙̦̫̪͓̠̺̫̖͙̫̏͂̒̽́̿̂̊́͂͋͜͠͝͝ṭ̴̜͎̮͉̙͍͔̜̾͋͒̓̏̉̄͘͠͝ͅę̷̡̭̹̰̺̩̠͓͌̃̕͜͝ͅͅx̵̧͍̦͈͍̝͖͙̘͎̥͕̾̾̍̀̿̔̄̑̈͝t̸̛͇̀̕
I have a fix, will open a PR :)
Related: https://github.com/JakeBecker/elixir-ls/issues/165 https://github.com/Microsoft/language-server-protocol/issues/376
Environment