request for comments

[ExpandingMan]

@kdheepak, @sbromberger, @davidanthoff, @non-Jedi

Hello all. I've been working a bit on using LanguageServer.jl in neovim. Most of the work to get us this far has been by @kdheepak, so let's all thank him for that.

Unfortunately, things are still not quite right. It seems to me that the principle problem with using LSP is that, while the actual language part of the protocol is explicitly specified, very little of install, initialization or shutdown steps are. (There is a single initialization spec in the protocol which passes very little useful information as far as logistics are concerned.) Because of this, I thought that we should "give up" on trying to make LanguageServer.jl "just work" everywhere, and instead admit to ourselves that we need a little bit of compatibility work.

To that end, I have created this very simple wrapper of LanguageServer.jl intended specifically for interacting with nvim-lsp, with the following goals

• Have a reasonable way of determining an appropriate environment directory (see here, this is a major existing issue that this package should solve) without putting any constraints on LanguageServer.jl
• Simplify the commands passed by Lua to Julia for starting up the language server. This will both allow us to do more elaborate things (e.g. the environment-checking behavior I've already implemented) and reduce the need for elaborate Lua scripts with elaborate embedded Julia command strings, which is not likely anyone's preference.
• Provide an isolated environment for LanguageServer to run in for neovim.
• Give us a clear place to provide any future neovim-specific functionality.

Unfortunately, because the specific interaction with LanguageServer.jl at startup is not standardized, and is given in the nvim-lsp repo, a coordinated effort is required, I don't see a good way right now of just letting everyone go ahead with whatever configuration they want. This means that we'd have to broadly agree whether this package is appropriate. I have created a fork of nvim-lsp which works with this package.

So, what does everyone think? Is this a good approach? I'm open to alternatives, and as you can see, not much effort needed to go into this so far. Again, because any real change would require changes to nvim-lsp, we need some kind of consensus. If we decide this package is a good idea, I'd be happy to move it to a github organization or grant admin rights to others.

[davidanthoff]

Because of this, I thought that we should "give up" on trying to make LanguageServer.jl "just work" everywhere, and instead admit to ourselves that we need a little bit of compatibility work.

Yes, that is certainly true: the idea for the LSP is that there will always be editor specific integration code that handles deployment, startup, configuration etc.

I don't think I have much more input on the nvim situation here, but I can point you to what we are doing in VS Code: this is the main startup script of the LS process. This is the code in the extension that spawns the LS process. We do some elaborate work to make sure the LS process is completely isolated from anything in the user's .julia folder, which has lead to a very stable deployment story for the LS in VS Code over the years.

The one thing that I took from the discussion in the other issues is that I think we should medium term attempt to remove the env keyword from LanguageServerInstance constructor, and probably also the custom julia/activateenvironment request, and instead try to use the normal config system for that value. But we won't touch that part until after Juliacon.

[kdheepak]

@davidanthoff, when a user switches projects in vscode, do you always start a new instance of LanguageServer.jl and kill the old one?

[kdheepak]

Thanks for creating this @ExpandingMan!

We don't need to fork nvim-lsp. I think users can create a custom nvim-lsp config in their vimrc, we can just provide instructions for how to do that. Once that is stable we can submit a PR to nvim-lsp to use that instead.

[davidanthoff]

when a user switches projects in vscode, do you always start a new instance of LanguageServer.jl and kill the old one?

If the user switches the "Julia Project" (i.e. in the status bar they select say the v1.4 env), then we send our custom request that a new environment was activated, see here. I think we should eventually replace that with a configuration setting update instead. Switching environment in that way will trigger a new run of the symbol server process if there are packages in the new environment that haven't been indexed yet.

Or did you mean what happens if a user opens say an entirely new VS Code window? In that case, VS Code spins up a new instance of the extensions, and it will spin up a new LS process. I.e. our LS process lifetime is tied to the extension lifetime and how VS Code manages that.

[kdheepak]

Thanks for the comment! I'm not sure what happens when someone opens a new file in neovim, and how that translates to telling the language server what to do. I imagine right now, if the user opens two different packages in the same instance of neovim right now, it'll only give information about one of the packages.

[davidanthoff]

The LS also supports multiple root folders, so the client could also use that, I guess...

[ExpandingMan]

So it sounds like there is some openness to having this package or something very like it as an interface between LanguageServer.jl and neovim, in particular David seems to be saying that we should expect this is needed.

Unless there are objections I will add some unit tests to this package (hard to test very much, but I like to have something in runtests.jl), add some documentation, register it, than make a PR to nvim-lsp. Of course PR's and suggestions are welcome if anyone feels something else is needed. At present, as far as I can tell using this package with my fork of nvim-lsp reproduces all functionality of the existing system and as an added benefit handles the environment selection somewhat more carefully.

One thing to resolve is that, at present, my fork of nvim-lsp simply clones this package rather than properly adding it through the package manager. The reason for this is that I very deliberately wanted this package to be able to manage its own environment, rather than having that environment created by the nvim-lsp lua script, but of course this requires the package to actually be cloned first. Since it is already common practice for vim plugins to clone from masters, I'm fine with this as a permanent solution, and I can always create a "release" branch for this that tracks tags. However, this is of course not a standard practice in the Julia community, so I'm not sure if everyone will be comfortable with this. (Though presumably nobody will object to cloning another branch if we can guarantee that it always tracks tags in the general registry.)

[ExpandingMan]

My PR to use this package by default. Please thumbs up/down it so the nvim-lsp maintainers know whether we are in agreement on this.