JSON and YAML lsp

[martinRenou]

Hello there,

We are interested in JSON-lsp and YAML-lsp that would work with the JupyterLab text editor.

I am opening an issue just to say that we might work on this, but I would like to know if folks here have some ideas/pointers to a starting point for this.

After a quick search I couldn't find an implementation for JSON-lsp, but I might not have searched properly. I only found this extension that is for SublimeText, I wonder if this could work out-of-the-box.

[krassowski]

Have you tried vscode-json-languageserver-bin and yaml-language-server documented in Language servers section of our docs? We do test against them and they work well AFAIK.

[krassowski]

You only need to install them with a findable node_modules. Though it is a pain that they require Node.js though...

[martinRenou]

Thanks a lot for the pointers, I will look into them.

You only need to install them with a findable node_modules

Which node_modules should it be included into?

[krassowski]

A discoverable one in a standard location, see node_roots. You can also add custom paths see extra_node_roots.

But I would love to be able to ship it without the node requirement. I think there are some options to package things as standalone (some kind of compilation that ships a stripped version of node), but I am slightly worried about the maintenance burden.

[krassowski]

The default in docs might not be clear, here is the code:

https://github.com/krassowski/jupyterlab-lsp/blob/161afed0254cb5b9467cd18cd350c1599135bf30/python_packages/jupyter_lsp/jupyter_lsp/types.py#L229-L256

[bollwyvl]

Things like YAML and JSON can probably live (and be faster than) a multi-hop round-trip to a proxy then server and back again.

The biggest win for supporting core jupyter functionality would be something that provided completion/better linting in the Advanced Settings editor, as there is already an a-priori schema.

Anyhow, to really start moving the ball forward in that space, we need some prototype approaches to #184.

Re: the other approach mentioned, personally, I won't be shipping/supporting a custom node or v8, that's for darned sure... but it's possible we could use @vercel/ncc to create true single-.js-file things that only require a node to exist, and not access to npm.com, a la how jlpm is shipped. Woe betide packages that think npm install during a build (pypi, conda or otherwise) is okay and going to work long-term. I've done this for other tools, like prettier, as well... it all depends on what binary nonsense they require. Anyhow, if these existed, it would be relatively easy to follow the jlpm wrapper approach so they worked predictably on windows/linux, given python (which isn't going anywhere right now for jupyter).

[bollwyvl]

And yet another approach: pygls 0.10.2 (there were a few hiccups) would now be my bet for starting greenfields language servers (selfishly, because I prefer a canonical, pure python pip package, with well-typed dependencies, in this case pydantic). A JSON/YAML/TOML/DHALL language server seems pretty doable, and the existing nodejs-based ones could be used as testing oracles. Language servers need more detail than some parsers provide, out-of-the-box, but lark exists, and can generate parsers that don't have a runtime dependency on lark itself... we've learned the hard way that the more dependencies a language server has, the harder it is to keep running, especially if you just care about the client and ease-of-install experience as we do on this project.

In another direction entirely: it's probably worth (somebody) investigating whether language server for language X written in language X|Y can be compiled to WASM and then shipped to the browser or run on the server, with python bindings. I want this for kernels, too! You gotta write your own file system, which is a bummer, but language servers often have weird needs, anyway.

[martinRenou]

And yet another approach: pygls 0.10.2 (there were a few hiccups) would now be my bet for starting greenfields language servers (selfishly, because I prefer a canonical, pure python pip package[...])

This looks to me like the more appealing approach.

[bollwyvl]

You write it, we'll ship a spec as an alternative to the nodejs one! Behavior if both available: undefined.

But seriously: if we ever get that top-level org (even naming it is hard), having "jupyter native" language servers (read: pure python, no vscode (or even jupyter*-lsp) cruft, aside from testing) living there would be very compelling. As they would be all statically typed, and speak a (under-defined) spec, the maintenance would be tractable.

[bollwyvl]

Semi-related: on some not-yet-open-source stuff for #551, we did have some success wrapping some existing well-packaged (1 or 2 files, no node_modules) nodejs-based servers  in python... something like this was sufficient to wire them up, along with some MANIFEST.in and entry_points cruft:

import sys, subprocess

def main():
    p = subprocess.Popen([NODE, PATH_TO_BIN_JS, *sys.argv[1:]], stdin=sys.stdin, stdout=sys.stdout)
    sys.exit(p.wait())

if __name__ == "__main__":
    main()

As the project we're doing that for is mostly interested in conda-based deployment, this would be sufficient for our needs, the node thing isn't much of an issue, and it's a whole heap easier than writing and supporting our own language servers. However, that being the case, it might make even more sense for us to handle that in conda with shell wrappers, where we can robustly define the node dependency, rather than pretending it's okay to do this kind of stuff with pip packages that are broken out of the box.

I don't have a good feel for how many of the current node-based servers we support would work this way... and we'd probably want them maintained separately from this client... though we could, again, support them upstream if they were packages for conda.

[martinRenou]

That looks like a good quick solution. Thanks a lot for sharing this.

[martinRenou]

But seriously: if we ever get that top-level org (even naming it is hard), having "jupyter native" language servers (read: pure python, no vscode (or even jupyter*-lsp) cruft, aside from testing) living there would be very compelling. As they would be all statically typed, and speak a (under-defined) spec, the maintenance would be tractable.

I'd be happy to start something on my side (whether it's the pygls approach or the wrapping approach), and move the project to this top-level org you're talking about as soon as it's created :)

[bollwyvl]

The sub-org is a long story, I wouldn't hold your breath there! Again, from my personal perspective, conda-forge is the right play for the node kind of thing vs pypi... most of the well-behaved ones could be noarch: generic, just shipping shell wrappers. Basically, on this repo, we could ship two specs per server, one that looks for the proper $PREFIX/bin/vscode-language-server-bin, with a fallback one that looks in node_modules.... wherever.

I'd very much like to see the pygls one happen, though!

[krassowski]

The sub-org is a long story, I wouldn't hold your breath there!

I mean we could do it right now, it is just better to finish the JEP started in #238 mostly to figure out if there are any suggestions on name of the organization and its scope. My passion for writing long pieces was diverted towards several academic things last month and I do not foresee myself having much time to work on this this month either, but anyone who wants to drive it in meantime have my full blessing.

[martinRenou]

I just created https://github.com/martinRenou/json-lsp which implements the wrapping of the npm package into a Python package. It seems to work as intended!

Would you have time to give it a quick look? Also, do you have an opinion on the Python package name? Not sure I made the ideal implementation here, I could make it by collecting information here and there in jupyterlab-lsp code/documentation.

I'd very much like to see the pygls one happen, though!

I agree! I needed a quick win though.