Support UTF-8 for the LSP protocol

[XVilka]

See the lengthy discussion here: https://github.com/microsoft/language-server-protocol/issues/376

It seems that dls is one of a few that uses UTF-16 https://github.com/Avi-D-coder/lsp-range-unit-survey

Would be nice if it will have UTF-8 support as well, at least optionally.

[LaurentTreguier]

Yes, I saw that discussion a while ago. I was planning on doing an almost complete rewrite of most of the server anyway, and this would be part of it

[WangWei90]

I don't if the following "invalid UTF-8 sequence Error" is connected with this.

void main()
{
    auto name = "章泽天";
}

[LaurentTreguier]

This could be related to this, if you use a language client plugin using UTF-8 instead of UTF-16. What editor and what plugin do you use ?

[WangWei90]

Really quick feedback :) I use vscode with "dlang" (of course, it's your plugin) on windows10, I have no settings related with "dlang".

[WangWei90]

How can I solve the "invalid UTF-8 sequence Error"? Everything works fine until I type some Chinese characters in my code.

[LaurentTreguier]

I tried copying the code example you gave aboce, but I don't have any issue with it. Does it happen if you copy and paste as well, or does it happen only as you type ?

[WangWei90]

I just copy the code to a new file and close everything else. Then, I do:

1. reload window
2. format document

[LaurentTreguier]

I can't reproduce this issue, even when explicitly formatting the file. DLS had problems with non-ascii characters for a while, but this should have been fixed some time ago. What version of DLS do you have ? (command: %LOCALAPPDATA%\dub\packages\.bin\dls-latest\dls.exe --version) The only thing I can think of right now is an old version of DLS that hasn't been able to update itself for some reason.

[WangWei90]

version ----------------- PS > ./dls.exe --version DLS version 0.26.2 [windows, x86_64, release, wininet] Compiled with LDC 1.20.1 (DMDFE: 2090)

vscode >> output >> DLS ------------- [Error - 上午9:20:13] Request textDocument/formatting failed. Message: Internal error Code: -32603 core.exception.UnicodeException@core\internal\utf.d(292): invalid UTF-8 sequence

0x00007FF75399732D 0x00007FF753997086 0x00007FF75398680C 0x00007FF75397610A 0x00007FF75397941F 0x00007FF753986C15 0x00007FF75397A1BE 0x00007FF75366975D 0x00007FF753719909 0x00007FF75364A3B6 0x00007FF753649CB3 0x00007FF753649407 0x00007FF7536B78CB 0x00007FF75367FD52 0x00007FF75367F98F 0x00007FF753978837 0x00007FF753978CC9