search

ebkalderon / tower-lsp

Language Server Protocol implementation written in Rust
Apache License 2.0
962 stars 54 forks source link

Question regarding server distribution #325

Closed TimJentzsch closed 2 years ago

TimJentzsch commented 2 years ago

Hey, I'm considering to rewrite a language server in Rust and plan to use tower-lsp for this.

My biggest question right now is how to distribute the server with the client, for example as VS Code extension. Do I need to compile to WASM to make it platform agnostic? Or do I have to compile several versions and choose the server binary dynamically?

So far I found #317 which provides a sample implementation, but I'm still a bit confused on how the distribution works.

Many thanks in advance!

ebkalderon commented 2 years ago

Thanks for the question! It really depends on how you personally would prefer to deploy your server.

One possible approach would be to ship the language server separately from the VSCode extension. Usually, it would be bundled together with your programming language's toolchain, but it could also be a separate package the users must install using cargo install or some other package manager. No matter the method, as long as the language server CLI binary is present on the system and can be talked to using standard I/O, the VSCode extension will work as expected. vscode_deno takes this approach, for example.

Another approach might be to have the VSCode extension check GitHub Releases and download the latest version of the language server to a temporary directory, if it doesn't already exist. So rather than blindly assuming the server is installed on the system, the VSCode extension can automatically download the latest version from the Internet as soon as the extension starts, and it can keep the server up to date over time.

Yet another approach might be to ship a pre-built version of the server bundled with the extension itself, whether it be with copies of native builds for all major platforms or a single cross-platform WASM build using wasm-pack with a form of IPC to communicate with the client. I have not dabbled with this approach myself, though, so I couldn't tell you precisely how this would be done.

Ultimately, vscode-languageclient offers a few different approaches for launching and connecting to a language server. It doesn't really care how the server binary was deployed to the system, just as long as it's there (unless it's embedded into the client extension as WASM and you're using some kind of IPC transport).

Unlike language client extensions, whose distribution method is relatively straightforward (Visual Studio Marketplace), the preferred distribution method for your server is entirely up to you, really.

TimJentzsch commented 2 years ago

Thank you for the fast and detailed response!

I'd prefer to avoid requiring the user to install the server separately, so I'll try out the WASM approach. wasm-pack looks very interesting for that, thanks for the link!

I think it would be valuable to add this to the documentation somewhere, maybe even with small example projects for some of the approaches. That would make it easier to get started with tower-lsp. :)

ebkalderon commented 2 years ago

No worries! Hope this information was helpful. I wish you the best of luck with your project.

I agree that it would be nice to document this information somewhere. Perhaps in the README? I think I'll open a new issue later today tracking this.

TimJentzsch commented 2 years ago

Sounds good! I'll see if I can set up a small example project/template with WASM for future reference.

TimJentzsch commented 2 years ago

One main limitation of this approach seems to be that tokio doesn't support WASM: tokio-rs/tokio#1597. So one would have to try to implement it with another runtime.

ebkalderon commented 2 years ago

Yes, that's precisely why the runtime-agnostic feature flag was added in v0.16.0 to decouple from tokio. Use a WASM-compatible executor with tower-lsp in that case.

TimJentzsch commented 2 years ago

Sorry for all the questions, but do you have experience with using async with WASM? Can you recommend any runtimes?

I'm currently trying out async-std, but it seems like the stdin and stdout are not available when compiling to WASM. Because they are required to create the Server struct I suspect that this will cause problems down the road.

ebkalderon commented 2 years ago

No worries, @TimJentzsch! Unfortunately, I do not have direct experience using embedded JS or WASM language servers in VSCode extensions (I've only ever worked with native binaries in my own projects). I think you may have to look up on your own which executors are available, though I'd certainly accept a pull request which adds a minimal WASM example to the examples folder!

However, I understand that vscode-languageclient provides multiple communication methods beyond standard I/O, including IPC and possibly WebSockets. If you check the documentation, the Server::new method doesn't actually require standard I/O support, but rather any valid AsyncRead/AsyncWrite pair. This means you should be able to use any WASM compatible transport, whether that be TCP/IP, WebSockets, etc. with tower-lsp, provided they satisfy these trait bounds.

TimJentzsch commented 2 years ago

Just to update you on this, I found out that basically all standard I/O methods are not available from the async libraries, because the target OS is unknown on WASM.

I also tried hard to manually implement AsyncRead/AsyncWrite for the nodejs stdin/stdout, but have run into a multitude of issues to the point where I think it might not be possible.

I'm definitely not proficient in neither Rust nor async, but I think that the WASM + async is not very mature at the moment so compiling a langauge server to WASM might not be feasible right now.

Do you have any resources/prior art/experience with the other approaches that don't require a separate user installation, i.e. dynamically downloading the server binary or bundling all server binaries and selecting the best one on runtime?

Many thanks for your assistance!