Implement a server in the compiler that serves information about the compilation

[andrewrk]

This will not be LSP, but a custom protocol just for Zig. A third party project can act as a proxy between this and LSP for that use case.

This issue covers both the compiler process (e.g. zig build-exe --listen=-) acting as a server, as well as a meta-protocol exposed by the build system (i.e. zig build --listen=-). The latter will act as a multiplexer for the former.

Once the initial implementation of both of these is created, follow-up issues can be made for specific features.

[binary132]

Does it make sense to tie this to the self-hosting Issue (https://github.com/zig-lang/zig/issues/89), or do you think separate pieces of the functionality required for LSP should be delegated into a pipeline of (caching?) components which perhaps the core compiler functionality would also rely on?

[andrewrk]

My plan is to solve this only in the self-hosted compiler.

[bb010g]

I'd be interested in investigating the differences between the LSP and what the Dart analyzer (API spec) provides, as JetBrains is actually able to use Dart's analyzer to fully power their Dart support due to the different features provided. Additionally, it looks like you can shim Dart's Analysis Server API to provide the Language Server Protocol without too much trouble.

[andrewrk]

I know @thejoshwolfe has some bones to pick with the language server protocol in particular. I think in the end we will likely support a custom protocol that is more powerful than LSP, and theoretically you could implement a proxy that consumes the former and provides the latter. Perhaps zig can support both.

[ghost]

I know @thejoshwolfe has some bones to pick with the language server protocol in particular.

No matter what, this will make working with zig so much easier 🥇

1. editor completion support but also the whole ecosystem

one more example: really nice code navigation https://about.sourcegraph.com/blog/sourcegraph-is-now-open-source/

[andrewrk]

This isn't contributor friendly yet. Lots to solve before work on this begins.

[pavildon]

@andrewrk is this your idea?: to have a language server implemented as a function in the std lib, so can be called from the build script. and then spawn the process with 'zig run-lsp ...args'. I think some basic methods can be implemented from the LSP spec with what zig have right now.

[andrewk]

You pinged the wrong account.

On Fri., 5 Jul. 2019, 12:54 pm Patricio Villalobos, < notifications@github.com> wrote:

@andrewk https://github.com/andrewk is this your idea?: to have a language server implemented as a function in the std lib, so can be called from the build script. and then spawn the process with 'zig run-lsp ...args'. I think some basic methods can be implemented from the LSP spec with what zig have right now.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/ziglang/zig/issues/615?email_source=notifications&email_token=AAAJIOXUZCHXUKAXEWGFXX3P52ZVDA5CNFSM4EECL752YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODZINCKI#issuecomment-508612905, or mute the thread https://github.com/notifications/unsubscribe-auth/AAAJIOWFIP67ELBFBJRWQVDP52ZVDANCNFSM4EECL75Q .

[ice1000]

I found this https://github.com/ice1000/intellij-zig It looks promising

Maybe not, hasn't been updated for a while

[ice1000]

Because I'm busy. As an alpha-stage language, Zig is changing its syntax from time to time, while the IntelliJ plugin parsers zig code using its own parser (created by me).

[ice1000]

I haven't implemented complicated reference resolution, thus you cannot have cross-file jumps and type inference. Zig is really unfriendly for IDE due to the C-interop hacks (you can use C references, which requires your IDE to understand C. For a language server, you also need to provide C source information, while you have LLVM in the Zig compiler to make things easier).

[xackus]

Please be aware that for best results the language parser should be error tolerant. https://code.visualstudio.com/api/language-extensions/language-server-extension-guide#error-tolerant-parser-for-language-server

Currently the JSON dumps @andrewrk suggested to use in #3429 are not generated in case of a compilation error. I do not know the compiler internals, how feasible is error tolerance when generating the JSON dump in stage1 and stage2 compiler?

Also, I presume the JSON format isn't planned to drastically change in the self-hosted compiler, is that correct? I would be happy to start writing the language server. In what directory in the repo should the server code be placed?

[ice1000]

You can don't have an error-tolerant parser. You can do: obtain AST each time when the code is well-formed, listen to programmer typing dots or explicitly invoke completion, and use the AST you obtained just now to provide completion.

Although this is definitely less good than error-tolerant parser (which is available in the unmaintained intellij-zig).

[xackus]

Correct. I just wanted make sure that new code is written to allow for error tolerance if possible.

[xackus]

I am currently implementing JSON-RPC.

[SuperAuguste]

If anyone is still actively looking for a language server, I'm one of the maintainers of zls, an almost full featured language server for Zig! It has local, global, and package completions and tons of other features such as goto, hover definitions, and document outlines coming soon.

You can check it out here.

[matklad]

This will not be LSP, but a custom protocol just for Zig. A third party project can act as a proxy between this and LSP for that use case.

Two thoughts:

1) consider building a custom library (with well-isolated pure-data shaped API) rather than an IPC protocol. LSP is all the rage nowadays, but better not to close the avenue for embedding the thing in process. 2) +1 on not coding against LSP specifically. The design principle we use for API layer in rust-analyzer is a) being 100% rust-specific b) targeting hypothetical “perfect” client which takes full advantage of what we provide. That general API is then explicitly dumbed down to LSP’s capabilities.

[Jarred-Sumner]

The current state as of January 2023 when you try to autocomplete sockaddr_t (from #14209)

Zig

1. Type std.os.sockaddr
2. No completions appear. Data types don't either. Go-to definition doesn't help

https://user-images.githubusercontent.com/709451/210762562-1d4befe6-ea46-4547-88bb-d8e9bf577d50.mp4

Rust

When you use Rust, completions work relatively consistently. In-editor feedback reports when the wrong number of arguments are passed to a function. It doesn't take more than a second or two to update the red squiggly lines. This improves developer productivity.

https://user-images.githubusercontent.com/709451/210764636-91f4c140-822b-4229-9d67-32f5fa6fb385.mp4

Go

It works pretty well, including go-to definition

https://user-images.githubusercontent.com/709451/210766932-f4d3432e-890c-4c89-a3bf-80350c04b454.mp4

[SuperAuguste]

@Jarred-Sumner Keep in mind that this is the current state of completions in zls and not the current state of completions in Zig; the compiler server is not implemented yet.

[Jarred-Sumner]

@Jarred-Sumner Keep in mind that this is the current state of completions in zls and not the current state of completions in Zig; the compiler server is not implemented yet.

Sure, but it is the current state of autocomplete for developers using Zig. It's relevant for this thread because it is an infrastructure/tooling problem that seems difficult for zls to solve well alone. rust-analyzer describes itself as a "Rust compiler frontend for IDEs" and that is essentially what this thread proposes (except, for Zig).

[matklad]

Some high-level thoughts on how one could actually do this:

https://matklad.github.io/2023/02/10/how-a-zig-ide-could-work.html

[andrewrk]

Thank you for sharing! This dives a bit deeper than my own musings on the topic, so I found it quite insightful.

If you think about IDE support for the first time, a very tantalizing idea is to try to lean just into incremental compilation.

Guilty as charged 😅 and excellent points thereafter. Although I do think that may be a nice starting point - especially with some of the upcoming compilation speed enhancements that I have in mind.