LanguageServer only works with VSCode

[onur]

Godot version: Godot Engine v3.2.beta4.official

OS/device including version: Debian GNU/Linux sid

Issue description:

I tried to use Godot Language Server with coc.nvim and LanguageClient-neovim clients. Godot is returning this error when clients sends initialize method:

ERROR: _parse_request: Not enough response headers, got: 1, expected >= 4.
   At: modules/websocket/wsl_server.cpp:49.

This is the assert line it's failing: wsl_server.cpp:49. Both clients are only sending Content-Lenght header and nothing else and godot is expecting more.

This is the request client is sending:

T 127.0.0.1:54458 -> 127.0.0.1:6008 [AP] #94
  Content-Length: 2601....                                                                                                                                                                  
##
T 127.0.0.1:54458 -> 127.0.0.1:6008 [AP] #96
  {"jsonrpc":"2.0","id":0,"method":"initialize","params":{"processId":1522387,"rootPath":"/home/onur/gamedev/04-tycoon","rootUri":"file:///home/onur/gamedev/04-tycoon","capabilities":{"wor
  kspace":{"applyEdit":true,"workspaceEdit":{"documentChanges":true,"resourceOperations":["create","rename","delete"],"failureHandling":"textOnlyTransactional"},"didChangeConfiguration":{"
  dynamicRegistration":true},"didChangeWatchedFiles":{"dynamicRegistration":true},"symbol":{"dynamicRegistration":true,"symbolKind":{"valueSet":[1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,1
  8,19,20,21,22,23,24,25,26]}},"executeCommand":{"dynamicRegistration":true},"configuration":true,"workspaceFolders":true},"textDocument":{"publishDiagnostics":{"relatedInformation":true},
  "synchronization":{"dynamicRegistration":true,"willSave":true,"willSaveWaitUntil":true,"didSave":true},"completion":{"dynamicRegistration":true,"contextSupport":true,"completionItem":{"s
  nippetSupport":true,"commitCharactersSupport":true,"documentationFormat":["markdown","plaintext"],"deprecatedSupport":true,"preselectSupport":true},"completionItemKind":{"valueSet":[1,2,
  3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25]}},"hover":{"dynamicRegistration":true,"contentFormat":["markdown","plaintext"]},"signatureHelp":{"dynamicRegistration":true
  ,"signatureInformation":{"documentationFormat":["markdown","plaintext"],"parameterInformation":{"labelOffsetSupport":true}}},"definition":{"dynamicRegistration":true},"references":{"dyna
  micRegistration":true},"documentHighlight":{"dynamicRegistration":true},"documentSymbol":{"dynamicRegistration":true,"symbolKind":{"valueSet":[1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,1
  8,19,20,21,22,23,24,25,26]}},"codeAction":{"dynamicRegistration":true,"codeActionLiteralSupport":{"codeActionKind":{"valueSet":["","quickfix","refactor","refactor.extract","refactor.inli
  ne","refactor.rewrite","source","source.organizeImports"]}}},"codeLens":{"dynamicRegistration":true},"formatting":{"dynamicRegistration":true},"rangeFormatting":{"dynamicRegistration":tr
  ue},"onTypeFormatting":{"dynamicRegistration":true},"rename":{"dynamicRegistration":true,"prepareSupport":true},"documentLink":{"dynamicRegistration":true},"typeDefinition":{"dynamicRegi
  stration":true},"implementation":{"dynamicRegistration":true},"declaration":{"dynamicRegistration":true},"colorProvider":{"dynamicRegistration":true},"foldingRange":{"dynamicRegistration
  ":true,"rangeLimit":5000,"lineFoldingOnly":true}}},"initializationOptions":{},"trace":"verbose","workspaceFolders":[{"uri":"file:///home/onur/gamedev/04-tycoon","name":"04-tycoon"}]}}   
#

cc @Faless

[Faless]

@onur I don't know much about the LSP, but the WebSocketServer, expects a valid websocket client handshake. A valid and minimal WebSocket handshake is like this:

GET /chat HTTP/1.1
Host: example.com:8000
Upgrade: websocket
Connection: Upgrade
Sec-WebSocket-Key: dGhlIHNhbXBsZSBub25jZQ==
Sec-WebSocket-Version: 13

Maybe your LanguageClient does not use websocket as a transport for LSP?

[onur]

@Faless that's the issue right there, Godot's Language Server is not actually a Language Server, it is a websocket server. According to LSP Specification a Language Server only supports two headers: Content-Lenght and Content-Type. IDK why godot requires a websocketserver on top of this.

None of the clients (except godot's vscode plugin) using a protocol like this to communicate a Language Server.

[Faless]

Again, not an expert on LSP, but for what I understand the protocol only specifies the message format (with one header) but does not really specify what kind of connection it uses. It could be raw tcp, websocket, a Unix pipe etc. The chosen connection for Godot was websocket. I can't say if that was the best choice. For what I see other editors have websocket support too, but again I don't know much about it. Maybe @Geequlim can help here.

On Sun, Dec 22, 2019, 19:12 Onur Aslan notifications@github.com wrote:

@Faless https://github.com/Faless that's the issue right there, Godot's Language Server is not actually a Language Server, it is a websocket server. According to LSP Specification https://microsoft.github.io/language-server-protocol/specifications/specification-3-14/ a Language Server only supports two headers: Content-Lenght and Content-Type. IDK why godot requires a websocketserver on top of this.

None of the clients (except godot's vscode plugin) using a protocol like this to communicate a Language Server.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/godotengine/godot/issues/34523?email_source=notifications&email_token=AAM4C3QVCQTDB6U6B7XADOLQZ6UZVA5CNFSM4J6IRXX2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEHPWSEY#issuecomment-568289555, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAM4C3TUFMGGGSHUAX2DJXTQZ6UZVANCNFSM4J6IRXXQ .

[Zylann]

Maybe websocket ended up being the choice because the subject was VSCode, which is built with web technology?

[Geequlim]

@Faless that's the issue right there, Godot's Language Server is not actually a Language Server, it is a websocket server. According to LSP Specification a Language Server only supports two headers: Content-Lenght and Content-Type. IDK why godot requires a websocketserver on top of this.

I think the LSP only specifies the message content format. It doesn't force implementations to use what kind of the connection method. I choose the websocket as it is quite simple for the implementation of both the server and the client (The VSCode extention). With websocket is is easy to allow multi-clients.

The LSP is designed to reduce the work of editor plugins which doesn't mean the client don't need to do anything.

[Geequlim]

And the GDScript Language Server is not work only with VSCode but also with Atom https://github.com/PrestonKnopp/language-gdscript

If your client does not support Websocket there are two main ways to connect to the GDScript language servser

1. Make a middleware for connecting to GDScript language service through websocket, and connect to the client through the methods supported by your editor.
2. If most of client support TCP connecting out of box. We can consider replacing the websocket server with a TCP server, but the existing clients must be refactored.

[habamax]

If your client does not support Websocket there are two main ways to connect to the GDScript language servser

That is sad. I was waiting for godot language server thinking I would be able to use it with vim.

I guess emacs folks will not be able to use it as well. (don't remember if emacs has websocket implementation)

PS, don't get me wrong, I use vim now for gdscript and will continue using it no matter if godot language server would be implemented in a websocket only way.

PPS, thanks for language server!

[Calinou]

@Geequlim How much work would it be to add LSP communication using HTTP requests? It's probably a bit slower than WebSocket, but it should be far easier to integrate into most editors. If the latency difference is large enough, we can keep both implementations.

[Geequlim]

@Calinou I'm not sure it is possible with HTTP because both the server and the client need to send messages to each other realtime

[habamax]

Not sure about HTTP, but there are language servers that does both stdio and socket communications. For example, solargraph of ruby https://github.com/castwide/solargraph

 Clients can connect using either stdio or TCP

I don't know how websocket is different with regular socket, though

[howdoicomputer]

emacs-lsp seems to only support stdio and TCP connections

https://github.com/emacs-lsp/lsp-mode/blob/6d4c54e869a6f43ab035d6ed91f58a45aab1bee1/lsp-mode.el#L974-L980

[NathanLovato]

I can confirm the issue. @ofrank123 told me that according to the Language Server specification, the server should accept messages starting with the Content-Length: header, but that the GDScript LSP server does not.

lsp-mode for Emacs does only support TCP right now, but the maintainer was saying he was okay with the idea of adding web socket support. On the other hand, most language servers I've seen out there communicate over TCP. So I'd expect also most editors to expect and support TCP rather than web-sockets by default.

[CyanBlob]

I think moving to TCP makes more sense, given how much more popular it seems for language clients. I could potentially work on this but I wouldn't expect to get anything done anytime soon. Unfortunately, I've been pretty busy recently

[NathanLovato]

If there's a consensus to move to TCP and there's someone who wants to take on that, I'd gladly sponsor the work through GDQuest. That is, up to some amount at least, depending on the scope of the task, as we are running on a limited budget.

@Geequlim @Faless may either of you be interested, or may you know someone who could do it?

[Geequlim]

@NathanLovato For me it is about two full day of work to replace it to raw tcp for the server and the vscode plugin. But I can't find time to do it before February 21.

[NathanLovato]

@Geequlim Thanks for the info. 🙂

@ofrank123 said he would get a fork and PR started for that moments ago.

[avahe-kellenberger]

Very excited to see this being looked into, thanks everyone. If there's a way to help sponsor/support the work specifically for the language server, I'd like to know.

[CyanBlob]

Work is being tracked in this PR: https://github.com/godotengine/godot/pull/35864 @ofrank123 seems to be almost finished with it. Their current PR doesn't support multiple clients, but we're not sure if anybody would mind losing that functionality or not. If you do, what's your use case?

[Faless]

What about running two instances of vim?

On Mon, Feb 3, 2020, 19:34 Andrew notifications@github.com wrote:

Work is being tracked in this PR: #35864 https://github.com/godotengine/godot/pull/35864 @ofrank123 https://github.com/ofrank123 seems to be almost finished with it. Their current PR doesn't support multiple clients, but we're not sure if anybody would mind losing that functionality or not. If you do, what's your use case?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/godotengine/godot/issues/34523?email_source=notifications&email_token=AAM4C3WRITSEDXTKQJ4CHWDRBBPS7A5CNFSM4J6IRXX2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEKU46DI#issuecomment-581553933, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAM4C3QZ2PD24D3HK5I2OFLRBBPS7ANCNFSM4J6IRXXQ .

[avahe-kellenberger]

Is that a common use case?

[CyanBlob]

@Faless Why do you need/want to run two instances of vim? Do other language servers allow this?

[habamax]

@Faless Why do you need/want to run two instances of vim? Do other language servers allow this?

Well this is quite common. I usually do have more than 1 vim instances open.

And yes other, I would say most of the servers are ok with it.

[NathanLovato]

What about running two instances of vim?

For vim it's normal to run multiple instances, e.g. use tmux or a program like tilix to have several tabs or sessions running in parallel, and to pop in and out of vim.

For emacs you typically use a daemon, as with many packages it becomes slow to load. But you can run several servers or open a single gui process as well. So no problem with that.

If that's enough to have two Godot projects open in parallel with completion, that sounds good to me.

[Faless]

I simply have multiple terminals tab and sometimes run multiple vim instances, this happens every day for me honestly...

On Mon, Feb 3, 2020, 20:17 Andrew notifications@github.com wrote:

@Faless https://github.com/Faless Why do you need/want to run two instances of vim? Do other language servers allow thi?s

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/godotengine/godot/issues/34523?email_source=notifications&email_token=AAM4C3WNFBEHW5NLGPFPUYDRBBUWNA5CNFSM4J6IRXX2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEKVBXEY#issuecomment-581573523, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAM4C3TYDZLCKBBPGVBASXTRBBUWNANCNFSM4J6IRXXQ .

[avahe-kellenberger]

I could see that when working on multiple projects at once.

[CyanBlob]

Interesting. Alright, well then I suppose it'd be worth keeping that functionality then. If @ofrank123 can't/doesn't want to work on that functionality, I could probably work on adding it sometime this week

[Faless]

I also don't see what is the problem in supporting multiple clients. The code was already in place, keep a reference to all the streampeers in a list, assign them a random id, poll them all in the loop, profit. There's reference implemention in wsl_server.cpp if you'd like some inspiration.

On Mon, Feb 3, 2020, 20:40 Fabio Alessandrelli < fabio.alessandrelli@gmail.com> wrote:

I simply have multiple terminals tab and sometimes run multiple vim instances, this happens every day for me honestly...

On Mon, Feb 3, 2020, 20:17 Andrew notifications@github.com wrote:

@Faless https://github.com/Faless Why do you need/want to run two instances of vim? Do other language servers allow thi?s

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/godotengine/godot/issues/34523?email_source=notifications&email_token=AAM4C3WNFBEHW5NLGPFPUYDRBBUWNA5CNFSM4J6IRXX2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEKVBXEY#issuecomment-581573523, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAM4C3TYDZLCKBBPGVBASXTRBBUWNANCNFSM4J6IRXXQ .

[CyanBlob]

With TCP sockets, I think using select() is a better approach than polling. Nobody is saying that supporting multiple clients would be complicated, just that somebody has to do it :)

[Faless]

Implementing select will be much harder, for little performance benefits in this case, but if someone wants to implement it in netsocket with a proper API that's fine. I would run that through a proposal first though.

On Mon, Feb 3, 2020, 21:05 Andrew notifications@github.com wrote:

With TCP sockets, I think using select() is a better approach than polling. Nobody is saying that supporting multiple clients would be complicated, just that somebody has to do it :)

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/godotengine/godot/issues/34523?email_source=notifications&email_token=AAM4C3WL5QEEI6HD5G2TCATRBB2HFA5CNFSM4J6IRXX2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEKVGSZA#issuecomment-581593444, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAM4C3UJLNGRTCQ5ZYPE3RTRBB2HFANCNFSM4J6IRXXQ .

[CyanBlob]

Ah, I didn't realize Godot natively supported TCP sockets with NetSocket and Tcp_Server. In that case, I would make use of those for consistency

[Faless]

You probably don't want to use netsocket directly, but its tcp abstraction (streampeertcp) which is also what's returned by tcp_server.take_connection()

On Mon, Feb 3, 2020, 21:25 Andrew notifications@github.com wrote:

Ah, I didn't realize Godot natively supported TCP sockets with NetSocket and Tcp_Server. In that case, I would make use of those for consistency

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/godotengine/godot/issues/34523?email_source=notifications&email_token=AAM4C3WKHL54SIZAWNU6D6LRBB4UPA5CNFSM4J6IRXX2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEKVIUYA#issuecomment-581601888, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAM4C3SGTD4AUAKERVLIG23RBB4UPANCNFSM4J6IRXXQ .

[CyanBlob]

I'm coming from the embedded world, so NetSocket already looked very abstracted and pleasant 😂 But you're right, keeping a collection of StreamPeerTCP to poll seems like the simplest way forward

[CyanBlob]

I have multiple clients working, but there's and issue with some of the communications between the server and my client:

[coc.nvim] error: Uncaught exception: Error: �^A^@^@Content-Length: 452^M
^M
{"jsonrpc":"2.0","method":"textDocument/publishDiagnostics","params":{"diagnostics":[{"code":5,"message":"The argument 'event' is never used in the function '_input'. If this is intended, prefix it with an underscore: '_event'","range":{"end":{"character":19,"line":26},"start":{"character":0,"line":26}},"severity":2,"source":"gdscript"}],"uri":"file:///home/andrew/Development/deckbuilding_roguelike/deckbuilding_roguelike/Scripts/Player/Player.gd"}}

Somewhere outside of gdscript_language_protocol.cpp is sending a malformed header. I'm having a hard time figuring out where that's coming from. If somebody can help find that, this should be just about ready to close. If the problem isn't clear, there are some garbage characters before the Content-Length header

[CyanBlob]

I inspected some of the traffic via Wireshark. There seem to be some malformed packets being sent. Some of them are marked as X11 packets, not TCP, and then are parsed wrong. For example, Content-Length: 369 got split in the middle. I'm running the Linux build. I can test Windows tomorrow most likely to see if I see the same behavior, but can somebody else test this if they get the time? The bad data comes in almost immediately after starting the server

[Faless]

At a quick glance, the problem is you are using put_string and not put_buffer to write to the socket. See docs: https://docs.godotengine.org/en/stable/classes/class_streampeer.html#class-streampeer-method-put-string

On Tue, Feb 4, 2020, 04:58 Andrew notifications@github.com wrote:

I inspected some of the traffic via Wireshark. There seem to be some malformed packets being sent. Some of them are marked as X11 packets, not TCP, and then are parsed wrong. For example, Content-Length: 369 got split in the middle. I'm running the Linux build. I can test Windows tomorrow most likely to see if I see the same behavior, but can somebody else test this if they get the time? The bad data comes in almost immediately after starting the server [image: Screenshot_20200203_215427] https://user-images.githubusercontent.com/5949523/73712559-15235380-46d0-11ea-968d-17d27471154f.png jjjjj

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/godotengine/godot/issues/34523?email_source=notifications&email_token=AAM4C3QRUVHKIPKYLOLWXXLRBDRW5A5CNFSM4J6IRXX2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEKWIORY#issuecomment-581732167, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAM4C3QS5XFNU3CIPOXLXKLRBDRW5ANCNFSM4J6IRXXQ .

[ofrank123]

Will change this asap, working on multiple clients right now.

[CyanBlob]

I just tested your new changes and it seems to be working great, multiple clients and all! Thanks for implementing this :) I'll go ahead and add instructions for coc.nvim, since this wasn't very clear to me: You need to add a custom language server (in CocConfig)

"languageserver": {
    "godot": {
      "host": "127.0.0.1",
      "filetypes": ["gd", "gdscript3"],
      "port": 6008
    }
  }

The filetype is not necessarily the same as the file extension. By default, vim doesn't assign a filetype to .gd files. You can set that up however you want, but this plugin does it for you, along with adding syntax highlighting: https://github.com/calviken/vim-gdscript3

[ofrank123]

Awesome! Glad to hear it's working for you! I might try messing with the VSCode and Atom plugins to see if I can get them to work, but no promises as I've never worked with those editors.

[0xh007]

Fantastic work everyone! I've been keeping an eye out for anything like this for probably a year now. Great to hear we've got it now. Thanks!

[Shatur]

@CyanBlob, it works for me too, but I also see [coc.nvim] Command: not found error after completion. Do you have this behavior too?

[mlvl42]

I was making a quick and dirty port of the vscode extension to coc.nvim when I stumbled upon this issue, switching from websocket to tcp is awesome news ! While we wait for the 3.2.2 release, perhaps this could help some vim users https://github.com/j3d42/coc-godot

[rafaeldelboni]

I've been using Godot + Nvim, since this merge #35864, and is being a lovely experience!

[Rybadour]

As a fellow Vim user this is excellent news! Thanks everyone!

[Calinou]

This was resolved in 3.2.2 by https://github.com/godotengine/godot/pull/35864, closing. Please create new issues in editor extensions' repositories if you can encounter any.

[TobiasDev]

I've been using Godot + Nvim, since this merge #35864, and is being a lovely experience!

@rafaeldelboni sorry to call you out like this, but I can't get auto-complete to work in my NVIM, would you mind sharing your how you got it to work?

I added this to my Exec Flags like stated in the documentation: "+call cursor({line}, {col})" {file} and changed Exec Path to my nvim-qt.exe.

Not sure if I need anything else? I have tried both with coc-godot installed and without it. Neither of them works for me. :(

[rafaeldelboni]

Hey hello Tobias, how you doing? No problem, could you double check your godot version is above 3.2.2 and if your coc-settings is looking similar to mine? Btw you could double check my init.nvim, I don't use coc-godot, just vannila coc.nvim and some other vim plugins.

I tested my setup yesterday with the current godot and everything was working fine :)

https://github.com/rafaeldelboni/dotfiles/blob/299e8a9f32446b272613ff1f133dc0b0db30b2fb/config/nvim/coc-settings.json#L12

[TobiasDev]

Hey hello Tobias, how you doing? No problem, could you double check your godot version is above 3.2.2 and if your coc-settings is looking similar to mine? Btw you could double check my init.nvim, I don't use coc-godot, just vannila coc.nvim and some other vim plugins.

I tested my setup yesterday with the current godot and everything was working fine :)

https://github.com/rafaeldelboni/dotfiles/blob/299e8a9f32446b272613ff1f133dc0b0db30b2fb/config/nvim/coc-settings.json#L12

Wow! Thanks! Using your CocConfing instantly fixed the issue for me. :) And even started showing me warnings and errors etc.!

Thank you once again! Now I can finally use Godot and NeoVim together!!! :) This made my day.

[khalid151]

I'm using neovim with coc-nvim and suggestions work but I get Command: not found upon completion (C-y).

Edit: This is a workaround for the time being, for my <CR> mapping:

inoremap <expr> <CR> complete_info().selected != -1 ?
            \ &filetype == "gdscript" ? (coc#expandable() ?  "\<C-y>" : "\<Esc>a") : "\<C-y>"
            \ : "\<C-g>u\<CR>"

[sQu1rr]

@khalid151 how does it work? In particular, what does the C-g u\ does? And what does coc#expandable() do (which didn't work for me)? I modified it to be

inoremap <expr> <CR> complete_info().selected != -1 ? "\<Esc>a" : "\<C-g>u\<CR>"

and placed it in after/ftplugin/gdscript.vim

[khalid151]

@sQu1rr It first checks if there's a completion menu open and there's an item selected with complete_info(). If there is, coc#expandable() will check if the current entry is a snippet to expand it. When there is no item selected and <CR> is needed, "\<C-g>u\<CR>" is run.

As I understood, <C-g>u is there to help with undo sequence. (here's the patch info that introduced this mapping)