search

ionide / FsAutoComplete

F# language server using Language Server Protocol
Other
409 stars 154 forks source link

Fails to startup on raspberrypi 4 running raspbian 64bit #871

Open bucketsize opened 2 years ago

bucketsize commented 2 years ago

Sourced precompiled version from vim-lsp-settings, not sure which version it is. Fails to start with very little diagnostic message.

$ dotnet .local/share/vim-lsp-settings/servers/fsautocomplete/fsautocomplete.netcore/fsautocomplete.dll --verbose

[15:55:31.355 INF] [Startup] Start - Ending LSP mode with ErrorStreamClosed

Same used to work fine with Manjaro 64bit running on raspberry pi4.

Anything further I can do to debug?

bucketsize commented 2 years ago

To further, what I'm running on currently is:

$ uname -a
Linux raspberrypi 5.10.63-v8+ #1459 SMP PREEMPT Wed Oct 6 16:42:49 BST 2021 aarch64 GNU/Linux
$ cat /etc/os-release 
PRETTY_NAME="Debian GNU/Linux 11 (bullseye)"
NAME="Debian GNU/Linux"
VERSION_ID="11"
VERSION="11 (bullseye)"
VERSION_CODENAME=bullseye
ID=debian
HOME_URL="https://www.debian.org/"
SUPPORT_URL="https://www.debian.org/support"
BUG_REPORT_URL="https://bugs.debian.org/"
$ lscpu
Architecture:                    aarch64
CPU op-mode(s):                  32-bit, 64-bit
Byte Order:                      Little Endian
CPU(s):                          4
On-line CPU(s) list:             0-3
Thread(s) per core:              1
Core(s) per socket:              4
Socket(s):                       1
Vendor ID:                       ARM
Model:                           3
Model name:                      Cortex-A72
Stepping:                        r0p3
CPU max MHz:                     1800.0000
CPU min MHz:                     600.0000
BogoMIPS:                        108.00
Vulnerability Itlb multihit:     Not affected
Vulnerability L1tf:              Not affected
Vulnerability Mds:               Not affected
Vulnerability Meltdown:          Not affected
Vulnerability Spec store bypass: Vulnerable
Vulnerability Spectre v1:        Mitigation; __user pointer sanitization
Vulnerability Spectre v2:        Vulnerable
Vulnerability Srbds:             Not affected
Vulnerability Tsx async abort:   Not affected
Flags:                           fp asimd evtstrm crc32 cpuid
baronfel commented 2 years ago

That message implies that your client sent the shutdown message to the LSP server (see here) - is there a way you can log LSP requests/responses from vim-lsp-settings?

vzarytovskii commented 2 years ago

I actually see the same behaviour on my laptop (Arch, x86_64), even when just running with dotnet ...fsautocomplete.dll (i.e. outside of any editor). Both Emacs and VSCode are not working too, FSAC just never loads.

It happens to both locally built FSAC as well as downloaded one.

vzarytovskii commented 2 years ago

Okay, so after some lightweight debugging and @baronfel's help, apparently FSAC tries to read headers (even first 2 bytes), and fails:

System.ArgumentException: The output byte buffer is too small to contain the encoded data, encoding 'Unicode (UTF-8)' fallback 'System.Text.EncoderReplacementFallback'. (Parameter 'bytes')
   at System.Text.Encoding.ThrowBytesOverflow()
   at System.Text.Encoding.ThrowBytesOverflow(EncoderNLS encoder, Boolean nothingEncoded)
   at System.Text.Encoding.GetBytesWithFallback(ReadOnlySpan`1 chars, Int32 originalCharsLength, Span`1 bytes, Int32 originalBytesLength, EncoderNLS encoder)
   at System.Text.Encoding.GetBytesWithFallback(Char* pOriginalChars, Int32 originalCharCount, Byte* pOriginalBytes, Int32 originalByteCount, Int32 charsConsumedSoFar, Int32 bytesWrittenSoFar, EncoderNLS encoder)
   at System.Text.Encoding.GetBytes(Char* pChars, Int32 charCount, Byte* pBytes, Int32 byteCount, EncoderNLS encoder)
   at System.Text.EncoderNLS.Convert(Char* chars, Int32 charCount, Byte* bytes, Int32 byteCount, Boolean flush, Int32& charsUsed, Int32& bytesUsed, Boolean& completed)
   at System.Text.Encoder.Convert(ReadOnlySpan`1 chars, Span`1 bytes, Boolean flush, Int32& charsUsed, Int32& bytesUsed, Boolean& completed)
   at System.IO.StdInReader.ReadLine(Byte[] buffer, Int32 offset, Int32 count)
   at System.ConsolePal.UnixConsoleStream.Read(Byte[] buffer, Int32 offset, Int32 count)
   at LanguageServerProtocol.LowLevel.readLine(Stream stream) in /home/u/code/fsac/src/LanguageServerProtocol/LanguageServerProtocol.fs:line 2023
   at LanguageServerProtocol.LowLevel.readHeaders(Stream stream) in /home/u/code/fsac/src/LanguageServerProtocol/LanguageServerProtocol.fs:line 2040
   at LanguageServerProtocol.LowLevel.read(Stream stream) in /home/u/code/fsac/src/LanguageServerProtocol/LanguageServerProtocol.fs:line 2056
   at LanguageServerProtocol.Server.start[a,b](FSharpMap`2 requestHandlings, Stream input, Stream output, FSharpFunc`2 clientCreator, FSharpFunc`2 serverCreator) in /home/u/code/fsac/src/LanguageServerProtocol/LanguageServerProtocol.fs:line 2835

This was when I'm sending Content-Type: 100\r\n header (100 can be any number really) Later on, catches exception here, and continues trying to read headers in a loop: image

However It doesn't fail, if i send smaller header Content-Ty: 100, in fact it hangs (and throws exception only if i send any string with length >=16)

vzarytovskii commented 2 years ago

Okay, it is very weird. I can't reproduce it on the same machine but with different kernel. Probably should wait until streamjsonrpc change is merged, and test again.

vzarytovskii commented 2 years ago

I could reproduce it in WSL on my windows machine too now.

I can try fixing it, or shall I wait for streamjsonrpc to be merged first and see if it fixes it? @baronfel @Krzysztof-Cieslak

baronfel commented 2 years ago

@vzarytovskii there's a C# LSP made by @razzmatazz on top of a shared LSP protocol implementation called csharp-ls that's distributed as a global tool. version 0.2.0 of that switched over to streamjsonrpc. you could try using that for C# and see if you get the hang as well? the hang should repro on csharp-ls 0.1.9 as well, because that version used the homegrown protocol implementation you've found the block in.

vzarytovskii commented 2 years ago

@vzarytovskii there's a C# LSP made by @razzmatazz on top of a shared LSP protocol implementation called csharp-ls that's distributed as a global tool. version 0.2.0 of that switched over to streamjsonrpc. you could try using that for C# and see if you get the hang as well? the hang should repro on csharp-ls 0.1.9 as well, because that version used the homegrown protocol implementation you've found the block in.

I've tried to repro it on csharp-ls@master, and it hangs on any input, regardless the length.

razzmatazz commented 2 years ago

Now that im thinking about this there is a somewhat curiously related issue with emacs-29 (but not with emacs-28):

One suspicion I have is that we have to set some options on stdin/stdout on startup. Maybe something related to pipe buffer size on Linux or something. As in all instances this happens on Linux

vzarytovskii commented 2 years ago

Now that im thinking about this there is a somewhat curiously related issue with emacs-29 (but not with emacs-28):

One suspicion I have is that we have to set some options on stdin/stdout on startup. Maybe something related to pipe buffer size on Linux or something. As in all instances this happens on Linux

I've seen it first in emacs too with fsac, however it reproduces outside just in console (no matter what she'll or locale settings). It also does not reproduce on a simple console app which just reads a line from the Console.GetStandardInput() (the stream csharp-ls uses).

razzmatazz commented 2 years ago

I was googling a bit and this rings bells to me: https://stackoverflow.com/questions/6081946/why-does-console-readline-have-a-limit-on-the-length-of-text-it-allows/6081967#6081967

I have released a new version of csharp-ls -- lets see if that fixes things for you, @vzarytovskii

.FSAC does the same thing in FsAutoComplete.Lsp.fs:startCore -- i.e. it does not pass buffer size parameter to Console.OpenStandardInput() and thus probably defaults to 256 bytes

@baronfel also note, that I could not find max JSON RPC message size, so currently I am using 64KB in csharp-ls.. but not sure what safe/correct value of this param should be..

razzmatazz commented 2 years ago

emacs@master problems looks to be something else, self inflicted pain :).. there is a follow-up on my investigation here

vzarytovskii commented 2 years ago

I was googling a bit and this rings bells to me: https://stackoverflow.com/questions/6081946/why-does-console-readline-have-a-limit-on-the-length-of-text-it-allows/6081967#6081967

I have released a new version of csharp-ls -- lets see if that fixes things for you, @vzarytovskii

* https://www.nuget.org/packages/csharp-ls/0.2.1

.FSAC does the same thing in FsAutoComplete.Lsp.fs:startCore -- i.e. it does not pass buffer size parameter to Console.OpenStandardInput() and thus probably defaults to 256 bytes

@baronfel also note, that I could not find max JSON RPC message size, so currently I am using 64KB in csharp-ls.. but not sure what safe/correct value of this param should be..

Alright, it seems to be all the same for me on the 0.2.1 - it just hangs on any input. It seems to be independent on shell, locale or user.

It is interesting that FSAC kinda continues reading if the initial string (e.g. header) is shorter than 16 characters.

Update: and FSAC still throws:

System.ArgumentException: The output byte buffer is too small to contain the encoded data, encoding 'Unicode (UTF-8)' fallback 'System.Text.EncoderReplacementFallback'. (Parameter 'bytes')
   at System.Text.Encoding.ThrowBytesOverflow()
   at System.Text.Encoding.ThrowBytesOverflow(EncoderNLS encoder, Boolean nothingEncoded)
   at System.Text.Encoding.GetBytesWithFallback(ReadOnlySpan`1 chars, Int32 originalCharsLength, Span`1 bytes, Int32 originalBytesLength, EncoderNLS encoder)
   at System.Text.Encoding.GetBytesWithFallback(Char* pOriginalChars, Int32 originalCharCount, Byte* pOriginalBytes, Int32 originalByteCount, Int32 charsConsumedSoFar, Int32 bytesWrittenSoFar, EncoderNLS encoder)
   at System.Text.Encoding.GetBytes(Char* pChars, Int32 charCount, Byte* pBytes, Int32 byteCount, EncoderNLS encoder)
   at System.Text.EncoderNLS.Convert(Char* chars, Int32 charCount, Byte* bytes, Int32 byteCount, Boolean flush, Int32& charsUsed, Int32& bytesUsed, Boolean& completed)
   at System.Text.Encoder.Convert(ReadOnlySpan`1 chars, Span`1 bytes, Boolean flush, Int32& charsUsed, Int32& bytesUsed, Boolean& completed)
   at System.IO.StdInReader.ReadLine(Byte[] buffer, Int32 offset, Int32 count)
   at System.ConsolePal.UnixConsoleStream.Read(Byte[] buffer, Int32 offset, Int32 count)
   at LanguageServerProtocol.LowLevel.readLine(Stream stream) in /home/u/code/fsac/src/LanguageServerProtocol/LanguageServerProtocol.fs:line 2023
   at LanguageServerProtocol.LowLevel.readHeaders(Stream stream) in /home/u/code/fsac/src/LanguageServerProtocol/LanguageServerProtocol.fs:line 2040
   at LanguageServerProtocol.LowLevel.read(Stream stream) in /home/u/code/fsac/src/LanguageServerProtocol/LanguageServerProtocol.fs:line 2056
   at LanguageServerProtocol.Server.start[a,b](FSharpMap`2 requestHandlings, Stream input, Stream output, FSharpFunc`2 clientCreator, FSharpFunc`2 serverCreator) in /home/u/code/fsac/src/LanguageServerProtocol/LanguageServerProtocol.fs:line 2835
vzarytovskii commented 2 years ago

Okay, after some more debugging and investigation, it for some reason tries to read a UTF8-sized character from ASCII stream. I could reproduce it on one more project. Same exception - fail to read more votes than there is in the stream. I'm not sure what exactly is causing it to fallback to the utf8 stream.

Edit: the project i've tested it on: https://github.com/faldor20/fsharp-language-server Stack trace:

Exception in read thread System.ArgumentException: The output byte buffer is too small to contain the encoded data, encoding 'Unicode (UTF-8)' fallback 'System.Text.EncoderReplacementFallback'. (Parameter 'bytes')
   at System.Text.Encoding.ThrowBytesOverflow()
   at System.Text.Encoding.ThrowBytesOverflow(EncoderNLS encoder, Boolean nothingEncoded)
   at System.Text.Encoding.GetBytesWithFallback(ReadOnlySpan`1 chars, Int32 originalCharsLength, Span`1 bytes, Int32 originalBytesLength, EncoderNLS encoder)
   at System.Text.Encoding.GetBytesWithFallback(Char* pOriginalChars, Int32 originalCharCount, Byte* pOriginalBytes, Int32 originalByteCount, Int32 charsConsumedSoFar, Int32 bytesWrittenSoFar, EncoderNLS encoder)
   at System.Text.Encoding.GetBytes(Char* pChars, Int32 charCount, Byte* pBytes, Int32 byteCount, EncoderNLS encoder)
   at System.Text.EncoderNLS.Convert(Char* chars, Int32 charCount, Byte* bytes, Int32 byteCount, Boolean flush, Int32& charsUsed, Int32& bytesUsed, Boolean& completed)
   at System.Text.Encoder.Convert(ReadOnlySpan`1 chars, Span`1 bytes, Boolean flush, Int32& charsUsed, Int32& bytesUsed, Boolean& completed)
   at System.IO.StdInReader.ReadLine(Span`1 buffer)
   at System.ConsolePal.UnixConsoleStream.Read(Span`1 buffer)
   at System.IO.ConsoleStream.ReadByte()
   at System.IO.BinaryReader.Read()
   at System.IO.BinaryReader.ReadChar()
   at LSP.Tokenizer.readLine(BinaryReader client) in /home/u/code/fsharp-language-server/src/LSP/Tokenizer.fs:line 38
   at LSP.Tokenizer.tokenize@56.GenerateNext(IEnumerable`1& next) in /home/u/code/fsharp-language-server/src/LSP/Tokenizer.fs:line 59
   at Microsoft.FSharp.Core.CompilerServices.GeneratedSequenceBase`1.MoveNextImpl() in D:\a\_work\1\s\src\fsharp\FSharp.Core\seqcore.fs:line 372
   at Microsoft.FSharp.Core.CompilerServices.GeneratedSequenceBase`1.System.Collections.IEnumerator.MoveNext() in D:\a\_work\1\s\src\fsharp\FSharp.Core\seqcore.fs:line 404
   at Microsoft.FSharp.Collections.Internal.IEnumerator.map@75.DoMoveNext(b& curr) in D:\a\_work\1\s\src\fsharp\FSharp.Core\seq.fs:line 77
   at Microsoft.FSharp.Collections.Internal.IEnumerator.MapEnumerator`1.System.Collections.IEnumerator.MoveNext() in D:\a\_work\1\s\src\fsharp\FSharp.Core\seq.fs:line 64
   at Microsoft.FSharp.Collections.SeqModule.TakeWhile@1308.GenerateNext(IEnumerable`1& next) in D:\a\_work\1\s\src\fsharp\FSharp.Core\seq.fs:line 1310
   at Microsoft.FSharp.Core.CompilerServices.GeneratedSequenceBase`1.MoveNextImpl() in D:\a\_work\1\s\src\fsharp\FSharp.Core\seqcore.fs:line 372
   at Microsoft.FSharp.Core.CompilerServices.GeneratedSequenceBase`1.System.Collections.IEnumerator.MoveNext() in D:\a\_work\1\s\src\fsharp\FSharp.Core\seqcore.fs:line 404
   at LSP.LanguageServer.connect@195.Invoke() in /home/u/code/fsharp-language-server/src/LSP/LanguageServer.fs:line 198
razzmatazz commented 2 years ago

Hey @vzarytovskii I think you are onto something.

I am not sure, but this sounds like environment variable issue... can you somehow force LANG=en_US.UTF-8 ? I think is from where .net selects console i/o encoding.

Otherwise we might just need to set Console.InputEncoding (and corresponding Console.OutputEncoding) on startup.

vzarytovskii commented 2 years ago

Hey @vzarytovskii I think you are onto something.

I am not sure, but this sounds like environment variable issue... can you somehow force LANG=en_US.UTF-8 ?

Yeah, that's the first thing I did (although I have en_GB.UTF8 as my locale). Also tried setting it to C, same issue.

I think is from where .net selects console i/o encoding.

Otherwise we might just need to set Console.InputEncoding (and corresponding Console.OutputEncoding) on startup.

I will try doing that.

vzarytovskii commented 2 years ago

Ok, setting Console.InputEncoding/Console.InputEncoding did not help, still fails with the same issue, but now shows the encoding which I've set for input stream.:

 System.ArgumentException: The output byte buffer is too small to contain the encoded data, encoding 'US-ASCII' fallback 'System.Text.EncoderReplacementFallback'. (Parameter 'bytes')
   at System.Text.Encoding.ThrowBytesOverflow()
   at System.Text.Encoding.ThrowBytesOverflow(EncoderNLS encoder, Boolean nothingEncoded)
   at System.Text.Encoding.GetBytesWithFallback(ReadOnlySpan`1 chars, Int32 originalCharsLength, Span`1 bytes, Int32 originalBytesLength, EncoderNLS encoder)
   at System.Text.ASCIIEncoding.GetBytesWithFallback(ReadOnlySpan`1 chars, Int32 originalCharsLength, Span`1 bytes, Int32 originalBytesLength, EncoderNLS encoder)
   at System.Text.Encoding.GetBytesWithFallback(Char* pOriginalChars, Int32 originalCharCount, Byte* pOriginalBytes, Int32 originalByteCount, Int32 charsConsumedSoFar, Int32 bytesWrittenSoFar, EncoderNLS encoder)
   at System.Text.Encoding.GetBytes(Char* pChars, Int32 charCount, Byte* pBytes, Int32 byteCount, EncoderNLS encoder)
   at System.Text.EncoderNLS.Convert(Char* chars, Int32 charCount, Byte* bytes, Int32 byteCount, Boolean flush, Int32& charsUsed, Int32& bytesUsed, Boolean& completed)
   at System.Text.Encoder.Convert(ReadOnlySpan`1 chars, Span`1 bytes, Boolean flush, Int32& charsUsed, Int32& bytesUsed, Boolean& completed)
   at System.IO.StdInReader.ReadLine(Span`1 buffer)
   at System.ConsolePal.UnixConsoleStream.Read(Span`1 buffer)
   at System.IO.ConsoleStream.ReadByte()
   at System.IO.BinaryReader.Read()
   at System.IO.BinaryReader.ReadChar()
   at LSP.Tokenizer.readLine(BinaryReader client) in /home/u/code/fsharp-language-server/src/LSP/Tokenizer.fs:line 38
   at LSP.Tokenizer.tokenize@56.GenerateNext(IEnumerable`1& next) in /home/u/code/fsharp-language-server/src/LSP/Tokenizer.fs:line 59
   at Microsoft.FSharp.Core.CompilerServices.GeneratedSequenceBase`1.MoveNextImpl() in D:\a\_work\1\s\src\fsharp\FSharp.Core\seqcore.fs:line 372
   at Microsoft.FSharp.Core.CompilerServices.GeneratedSequenceBase`1.System.Collections.IEnumerator.MoveNext() in D:\a\_work\1\s\src\fsharp\FSharp.Core\seqcore.fs:line 404
   at Microsoft.FSharp.Collections.Internal.IEnumerator.map@75.DoMoveNext(b& curr) in D:\a\_work\1\s\src\fsharp\FSharp.Core\seq.fs:line 77
   at Microsoft.FSharp.Collections.Internal.IEnumerator.MapEnumerator`1.System.Collections.IEnumerator.MoveNext() in D:\a\_work\1\s\src\fsharp\FSharp.Core\seq.fs:line 64
   at Microsoft.FSharp.Collections.SeqModule.TakeWhile@1308.GenerateNext(IEnumerable`1& next) in D:\a\_work\1\s\src\fsharp\FSharp.Core\seq.fs:line 1310
   at Microsoft.FSharp.Core.CompilerServices.GeneratedSequenceBase`1.MoveNextImpl() in D:\a\_work\1\s\src\fsharp\FSharp.Core\seqcore.fs:line 372
   at Microsoft.FSharp.Core.CompilerServices.GeneratedSequenceBase`1.System.Collections.IEnumerator.MoveNext() in D:\a\_work\1\s\src\fsharp\FSharp.Core\seqcore.fs:line 404
   at LSP.LanguageServer.connect@195.Invoke() in /home/u/code/fsharp-language-server/src/LSP/LanguageServer.fs:line 198

Edit: as mentioned above, now I can reproduce it on both full linux installation and WSL2 from both my work laptop and home PC.