pyright-langserver

[jdtsmith]

Description

I'm trying to configure pyright as a Python language server, but haven't had much success.

Reproduce

So far I have managed to get the jupyterlab-lsp installed (after struggling with some Internal Server 500 errors):

% jupyter labextension list             
JupyterLab v3.0.7
Other labextensions (built into JupyterLab)
   app dir: /home/jdsmith/.local/share/jupyter/lab
        @krassowski/completion-theme v2.0.0 enabled OK
        @krassowski/jupyterlab-lsp v3.4.1 enabled OK

Uninstalled core extensions:
    @krassowski/jupyterlab-lsp

But I don't get any additional completion/underlining/etc. features.

Expected behavior

I hoped to see underlines and signatures/etc. provided by pyright.

Context

• Language Server: pyright-langserver
• Language Server version: 1.1.113
• Language Server installed with: npm
• Language Server Spec

I have also added the following file as pyright.json to /etc/jupyter/jupyter_notebook_config.d

{
  "pyright-langserver": {            # the name of the implementation
      "version": 1,                      # the version of the spec schema
      "argv": ["pyright-langserver", "--stdio"],  # a list of command line arguments
      "languages": ["python"]            # a list of languages it supports
  }
}

I'm not certain if this is the right location to implement a new server spec. In fact I found the docs lacking there: lots of descriptions of what might go in such a spec file to configure a new LSP server, but no mention I could find of where to put it!

Before I dump a bunch of files on you, I want to see how I might learn whether juptyerlab_lsp is in fact running, and if it has found (and rejected?) the langserver spec. I looked for [lsp] output in a --debug run of jupyter lab, but did not see any mentions.

[krassowski]

It's likely jupyter_server_config not jupyter_notebook_config if you are using JupyterLab 3.0 (unless using JupyterHub/Binder where it might be one or the other). jupyter lab --debug should show a message like: [lsp] The following Language Servers will be available:; if it is not there, the specs were not picked up.

Please do let us know if you succeed or need more help.

[krassowski]

Also, we are at version 2 of the specs, I think that you are looking at outdated documentation version. Please see here: https://jupyterlab-lsp.readthedocs.io/en/latest/Configuring.html

Do we have outdated instructions elsewhere?

[jdtsmith]

Thanks for the speedy response. I patterned the spec off the python one here, but I see from the other link it's been updated. It might be nice in the new SPEC format docs to mention it's new and what changes were made, and provide some guidance on where you might put such a spec file so it could be picked up.

I found .local/etc/jupyter/jupyter_server_config.d/ and placed a newly formatted spec file there, but it did not pick it up. Debug info does include [LabApp] JupyterLab application directory is /home/jdsmith/.local/share/jupyter/lab but apparently that doesn't apply to the server.

Since my default jupyter is installed by a system package (Fedora33), I suspected maybe that directory isn't being consulted, so I made a new directory /etc/jupyter/jupyter_server_config.d/, and put my newly formatted spec file there. I can tell it's being read, because I accidentally left comments in it (which may be confusing to have in a json file example). I think any of the "system-wide" locations should work (I tried a few), and I see things like:

[ServerApp] Paths used for configuration of jupyter_server_config: 
<snip>  
        /usr/local/etc/jupyter/jupyter_server_config.d/pyright.json

But after moving them and fixing the json syntax, still no [lsp] debug mentions to report. Here's the new spec:

{
  "LanguageServerManager": {
    "language_servers": {
      "pyright-language": {
        "version": 2,
    "argv": ["pyright-langserver", "--stdio"],
    "languages": ["python"]
      }
    }
  }
}

EDIT: Incremented version to 2, still no luck. I do see some 404 errors pertaining to lsp:

[W 2021-02-19 21:58:55.108 ServerApp] 404 GET /lab/api/settings/@krassowski/jupyterlab-lsp:plugin?1613789935081 (127.0.0.1): Schema not found: /usr/local/share/jupyter/lab/schemas/@krassowski/jupyterlab-lsp/plugin.json
[W 2021-02-19 21:58:55.108 LabApp] Schema not found: /usr/local/share/jupyter/lab/schemas/@krassowski/jupyterlab-lsp/plugin.json
[W 2021-02-19 21:58:55.109 LabApp] 404 GET /lab/api/settings/@krassowski/jupyterlab-lsp:plugin?1613789935081 (127.0.0.1) 8.02ms referer=http://localhost:8890/lab/tree/code/python/simla/Untitled5.ipynb

[krassowski]

Just to make sure: are you seeing some [lsp] messages when using --debug? Like [lsp] rootUri will be ...? If not, are you sure that you have the jupyter_lsp installed? What is your jupyter serverextension list?

[krassowski]

Also this:

Uninstalled core extensions:
    @krassowski/jupyterlab-lsp

Looks like an issue during installation. Could you remove @krassowski/jupyterlab-lsp and then remove build_config.json as described here and in https://github.com/jupyterlab/jupyterlab/issues/8122#issuecomment-606146683 and then isntall from pip/conda:

pip install jupyterlab-lsp

Please not it is jupyterlab-lsp here (while I previously asked about jupyter-lsp; jupyterlab-lsp will install both).

[jdtsmith]

Hmmm...:

% jupyter serverextension list
config dir: /usr/local/etc/jupyter
    jupyterlab  enabled 
    - Validating...
      jupyterlab 3.0.7 OK
~
% jupyter labextension list
JupyterLab v3.0.7
/home/jdsmith/.local/share/jupyter/labextensions
        @krassowski/jupyterlab-lsp v3.4.1 enabled OK (python, jupyterlab_lsp)

Other labextensions (built into JupyterLab)
   app dir: /usr/local/share/jupyter/lab

[krassowski]

jupyter serverextension list should include:

    jupyter_lsp  enabled 
    - Validating...
      jupyter_lsp 1.1.3 OK

but your is missing. Please do uninstall the npm-installed version (and remove the build file as described above) and follow the current installation instructions, as detailed in README,

[jdtsmith]

npm was only used for pyright. I removed @krassowski/jupyterlab-lsp and build_config.json, then:

 pip install 'jupyterlab>=3.0.0,<4.0.0a0' jupyterlab-lsp

which proceeds to install without incident in my user .local directory. But it turns out the jupyter installed by the system apparently doesn't look there. So I did a sudo pip install (ignoring the warning against that), and:

% jupyter serverextension list                                
config dir: /usr/local/etc/jupyter
    jupyter_lsp  enabled 
    - Validating...
      jupyter_lsp 1.1.3 OK
    jupyterlab  enabled 
    - Validating...
      jupyterlab 3.0.7 OK

But despite that, still no language servers reported (and the Jupyter Lab interface tells me Missing: 1 python at bottom status):

[D 2021-02-19 22:23:08.966 ServerApp] Paths used for configuration of jupyter_server_config: 
        /etc/jupyter/jupyter_server_config.d/pyright.json
...
[D 2021-02-19 22:23:09.237 ServerApp] [lsp] rootUri will be file:///home/jdsmith
[D 2021-02-19 22:23:09.238 ServerApp] [lsp] virtualDocumentsUri will be file:///home/jdsmith/.virtual_documents
[D 2021-02-19 22:23:09.238 ServerApp] [lsp] The following Language Servers will be available: {}
[I 2021-02-19 22:23:09.238 ServerApp] jupyter_lsp | extension was successfully loaded.

Anyway, some progress!

[krassowski]

Good! I think that it might be the issue of finding the pyright-langserver now (sadly I am not sure if we show detected schema that have no visible executables; this needs a PR). Some standard troubleshooting would include:

• Is this a globally visible executable? Or, is it a command that is only available when in specific directory (like npx-discoverable?)
• Does it work with an absolute path to it?
• What does which pyright-langserver return?

[jdtsmith]

Yeah it's on the path in /usr/local/bin/ But hard-coding path like:

    "argv": ["/usr/local/bin/pyright-langserver", "--stdio"],

does not help. Here it is running from the command line (same user which starts jupyter lab):

% /usr/local/bin/pyright-langserver --stdio   
Content-Length: 119

{"jsonrpc":"2.0","method":"window/logMessage","params":{"type":3,"message":"Pyright language server 1.1.113 starting"}}Content-Length: 143

{"jsonrpc":"2.0","method":"window/logMessage","params":{"type":3,"message":"Server root directory: /usr/local/lib/node_modules/pyright/dist/"}}

[krassowski]

I can reproduce the issue. The json files are not being picked up from the config directories :fearful: CC @bollwyvl

The workaround is to place the json content in jupyter_server_config.json file, for example: /etc/jupyter/jupyter_server_config.json. Can you confirm that it works for you?

[jdtsmith]

Yes!

[lsp] The following Language Servers will be available: {
      "pyright-language": {
        "argv": [
          "/usr/local/bin/pyright-langserver",
          "--stdio"
        ],
        "languages": [
          "python"
        ],
        "version": 2
      }
    }

Seems to be doing something; I get underlines and can hold Ctrl there to see a small pop-up (like numpy module), but most things are unknown. I tried a rename and it failed with null is not an object (the error flew by). But at least it's working... If there's additional logging that would put pyright through its paces you can suggest happy to try it out.

[krassowski]

That's great! I will give it a go tomorrow. It would be interesting to hear how autocompletion works. Thanks for your persevere and willingness to try out new things. It was not only good to see that pyright can work with this extension but also great that we discovered this issue band can now look for ways to fix it.

Please feel welcome to open new issues if you encounter any problems, and to contribute patches whether to code or to the documentation.

[jdtsmith]

Great. Autocompletion seems fairly good if a bit slow, but item recognition is not great, and signatures don't come through. I was actually interested in testing the quality of pyright in jupyterlab to check against Emacs lsp-mode and lsp-pyright since the latter seem to know some but not most numpy functions, for example. But it seems some of of the LSP data isn't making it through. I wonder if pyright actually supports scanning ipynb files? I know pylance does (which is the fancy can-only-use-with VSCode LSP new Microsoft server, sigh).

[yuntan]

I found that setting python.analysis.useLibraryCodeForTypes to true works for popups.

default	useLibraryCodeForTypes: true

Open "Advanced Settings Editor" > "Language Server" and edit "User Preferences".

{
    "language_servers": {
        "pyright": {
            "serverSettings": {
                "python.analysis.useLibraryCodeForTypes": true
            }
        }
    }
}

Then restart Jupyter.

Note Starting with numpy 1.20 type stubs are included in numpy. Pyright uses type stubs for completion and popups. useLibraryCodeForTypes: true is effective for packages without type stubs (like pandas).

[krassowski]

Closing as pyright auto-detection was added in #587 and will be available in the next version :)

[nyngwang]

This is a dagger.