search

sublimelsp / LSP-pyright

Python support for Sublime's LSP plugin provided through microsoft/pyright.
MIT License
126 stars 13 forks source link

How to make stdlib stubs work within a virtualenv #325

Closed mikez closed 4 months ago

mikez commented 4 months ago

(Discussion moved from here.)

Aim: show lsp_hover docstrings for both pip-installed modules and stdlibs. To achieve the latter, I made pyi-stubs. This works. 👌

However, once I switch to a virtual environment, that is, provide the venvPath and venv options in pyrightconfig.json, then the stdlib docstrings disappear. Everything else works: the packages installed in the virtual environment show their docstrings, as do the custom editable installs I put in the extraPaths (pyright doesn't seem to pick those up natively otherwise). It is only the pyi-stubs of the stdlib which don't show up.

mikez commented 4 months ago

BTW: If there are ways in which I can easily debug and troubleshoot this myself, I'm happy to do so. Still a newcomer to this project and not quite sure where to start.

jfcherng commented 4 months ago

It seems that pyright prefers its own typeshed stubs for stdlib but I am not sure. Set python.analysis.logLevel to Trace will provide more information for debugging. But since I don't know what's the expected output, I can't interpret them.

One thing works on my test is put stdlib stubs into typings/ under project root, but we don't want to do that for every project for sure...

One person may have a same issue: https://github.com/microsoft/pyright/issues/7016#issuecomment-2042476674 But in my local test I randomly picked some pyright versions (even back to 1.1.320), none of them works for stdlib stubs in extraPaths. The one in bundled typeshed seems to be always preferred. But in Pylance, custom stdlib stubs in extraPaths works.

I have no idea what should happen from the debug output, given I don't know how pyright works internally. We, LSP-pyright, have no related change. I suggest ask pyright. At least, I can't help and I don't think this is a LSP-pyright issue since LSP-pyright does pass user settings to pyright, but it doesn't work as expected.

mikez commented 4 months ago

@jfcherng python.analysis.logLevel in the LSP-pyright.sublime-settings: should I see this output under "LSP: Toggle Log Panel" or elsewhere? I don't see any difference right now between "Information" and "Trace" in that Log Panel output.

jfcherng commented 4 months ago

@jfcherng python.analysis.logLevel in the LSP-pyright.sublime-settings: should I see this output under "LSP: Toggle Log Panel" or elsewhere? I don't see any difference right now between "Information" and "Trace" in that Log Panel output.

Either in LSP's Log panel or ST console. Depending on LSP version you used.

mikez commented 4 months ago

I looked into the logs. It seems in all cases this directory is loaded:

~/Library/Caches/Sublime Text/Package Storage/LSP-pyright/21.7.1/language-server/node_modules/pyright/dist/typeshed-fallback/stdlib

and the custom stdlib-pyi directory is listed as part of the extraPaths in the logs but then not considered.

  1. When I have venv enabled, then it ignores the docstrings of built-ins all-together, including os and json which have docstrings as shipped with Python. ❌
  2. When I don't have venv enabled, then it reads the docstrings from the source (e.g. os.py and json/__init__.py) where its present ✅; and where not (e.g. str.strip) it shows no docstrings. ❌
  3. When I manually edit ~/Library/Caches/Sublime Text/Package Storage/LSP-pyright/21.7.1/language-server/node_modules/pyright/dist/typeshed-fallback/stdlib/builtins.pyi and add docstrings to strip, then it shows up correctly. ✅

So the only very hacky solution I can think of right now is to overwrite … /typeshed-fallback/stdlib/builtins.pyi and the like with versions with docstrings; it seems there must be a better way?

jfcherng commented 4 months ago

A finding is that, by revising the extension of our stdlib stubs from .pyi to .py, Pyright seems to consider they are source codes with docstrings and adds typeshed's type annotations to them in the output. Thus we have perfect results.

stdlib stubs with .py extension for Python 3.12

mikez commented 4 months ago

Thanks @jfcherng. I opened a discussion at pyright. It seems to me, pyright should consider including native support for builtin docstrings as manual generation seems somewhat difficult to maintain.

jfcherng commented 4 months ago

I think https://github.com/sublimelsp/LSP-pyright/issues/325#issuecomment-2085239473 is 99% perfect, unless you want to see stdlib's real source codes.

Or, I have a gift for you if you can contact me on GitHub via jfcherng.