search

microsoft / vscode-python

Python extension for Visual Studio Code
https://aka.ms/pvsc-marketplace
MIT License
4.32k stars 1.18k forks source link

Jedi features fail across notebook cells #24347

Open tmchartrand opened 5 days ago

tmchartrand commented 5 days ago

Type: Bug

Behaviour

When using the Jedi language server, all LS features work properly in a single cell, but don't recognize variables defined in one cell and used in another. Go to def gives a "no definition found" and hover previews are not available (autocomplete does still work).

It seems to me that the fix for microsoft/vscode-python#23897 worked partially but not completely - I think I missed this when I first tested that fix due to only testing features in a single cell initially.

Steps to reproduce:

  1. Change the python language server setting to jedi.
  2. Create a jupyter notebook with two cells, run import os in the first cell and type os in the second cell.
  3. Note that LS features work for os in the first cell and not the second.
  4. Add import os additionally to the second cell - features work there again.

Diagnostic data

Nothing seen in the logs, but I'm not sure how to change the Python extension logging level

Output for Python in the Output panel (View→Output, change the drop-down the upper-right of the Output panel to Python)

2024-10-25 13:17:37.279 [info] Starting Jedi language server for aind-data-schema. 2024-10-25 13:27:56.724 [info] > . ./.venv/bin/activate && echo 'e8b39361-0157-4923-80e1-22d70d46dee6' && python ~/.vscode/extensions/ms-python.python-2024.16.1-linux-x64/python_files/printEnvVariables.py 2024-10-25 13:27:56.724 [info] shell: bash

Extension version: 2024.16.1 VS Code version: Code 1.94.2 (384ff7382de624fb94dbaf6da11977bba1ecd427, 2024-10-09T16:08:44.566Z) OS version: Linux x64 5.14.0-427.31.1.el9_4.x86_64 Modes:

User Settings

``` condaPath: "" languageServer: "Jedi" ```

Installed Extensions |Extension Name|Extension Id|Version| |---|---|---| |autoDocstring - Python Docstring Generator|njpwerner.autodocstring|0.6.1| |Black Formatter|ms-python.black-formatter|2024.2.0| |Data Wrangler|ms-toolsai.datawrangler|1.0.2| |Git Graph|mhutchie.git-graph|1.30.0| |GitHub Codespaces|GitHub.codespaces|1.16.19| |GitHub Copilot|GitHub.copilot|1.241.0| |GitHub Copilot Chat|GitHub.copilot-chat|0.21.2| |GitHub Pull Requests|GitHub.vscode-pull-request-github|0.98.0| |GitLens — Git supercharged|eamodio.gitlens|15.6.2| |isort|ms-python.isort|2023.10.1| |JavaScript Debugger|ms-vscode.js-debug|1.94.0| |JavaScript Debugger Companion Extension|ms-vscode.js-debug-companion|1.1.3| |Jupyter|ms-toolsai.jupyter|2024.9.1| |Jupyter Cell Tags|ms-toolsai.vscode-jupyter-cell-tags|0.1.9| |Jupyter Keymap|ms-toolsai.jupyter-keymap|1.1.2| |Jupyter Notebook Renderers|ms-toolsai.jupyter-renderers|1.0.19| |Jupyter Slide Show|ms-toolsai.vscode-jupyter-slideshow|0.1.6| |Live Share|ms-vsliveshare.vsliveshare|1.0.5941| |Markdown All in One|yzhang.markdown-all-in-one|3.6.2| |Pylance|ms-python.vscode-pylance|2024.10.1| |Python|ms-python.python|2024.16.1| |Python Debugger|ms-python.debugpy|2024.12.0| |Python Environment Manager|donjayamanne.python-environment-manager|1.2.4| |Rainbow CSV|mechatroner.rainbow-csv|3.11.0| |Remote - SSH|ms-vscode-remote.remote-ssh|0.107.1| |Remote - SSH: Editing Configuration Files|ms-vscode-remote.remote-ssh-edit|0.86.0| |Remote Explorer|ms-vscode.remote-explorer|0.4.1| |Table Visualizer for JavaScript Profiles|ms-vscode.vscode-js-profile-table|1.0.10| |YAML|redhat.vscode-yaml|1.14.0|
System Info |Item|Value| |---|---| |CPUs|Intel(R) Core(TM) i7-6700 CPU @ 3.40GHz (8 x 0)| |GPU Status|2d_canvas: enabled
canvas_oop_rasterization: enabled_on
direct_rendering_display_compositor: disabled_off_ok
gpu_compositing: enabled
multiple_raster_threads: enabled_on
opengl: enabled_on
rasterization: enabled
raw_draw: disabled_off_ok
skia_graphite: disabled_off
video_decode: enabled
video_encode: disabled_software
vulkan: disabled_off
webgl: enabled
webgl2: enabled
webgpu: disabled_off
webnn: disabled_off| |Load (avg)|0, 1, 1| |Memory (System)|30.98GB (19.81GB free)| |Process Argv|--crash-reporter-id 549ec349-1818-4235-911a-668a77cf10fe| |Screen Reader|no| |VM|0%| |DESKTOP_SESSION|gnome| |XDG_CURRENT_DESKTOP|GNOME| |XDG_SESSION_DESKTOP|gnome| |XDG_SESSION_TYPE|wayland|
A/B Experiments ``` vsliv368cf:30146710 vspor879:30202332 vspor708:30202333 vspor363:30204092 vscod805:30301674 binariesv615:30325510 vsaa593cf:30376535 py29gd2263:31024239 c4g48928:30535728 azure-dev_surveyone:30548225 2i9eh265:30646982 962ge761:30959799 pythongtdpath:30769146 pythonnoceb:30805159 asynctok:30898717 pythonmypyd1:30879173 2e7ec940:31000449 pythontbext0:30879054 cppperfnew:31000557 dsvsc020:30976470 pythonait:31006305 dsvsc021:30996838 0ee40948:31013168 dvdeprecation:31068756 dwnewjupytercf:31046870 2f103344:31071589 impr_priority:31102340 nativerepl1:31139838 refactort:31108082 pythonrstrctxt:31112756 wkspc-onlycs-t:31132770 wkspc-ranged-t:31151552 cf971741:31144450 autoexpandse:31146404 iacca2:31156134 notype1cf:31157160 5fd0e150:31155592 dwcopilot:31164048 iconenabled:31158251 ```
tmchartrand commented 5 days ago

Also note that as in microsoft/vscode-python#23897, this issue is not present in older extension versions: jupyter v2023.9.100, python v2024.2.1

karthiknadig commented 3 days ago

This has not changed for a while in the python extension. Moving this to Jupyter for further investigation.

tmchartrand commented 2 days ago

@DonJayamanne curious for your thoughts on how this relates to the recent "middleware addon component" change - perhaps unrelated, but that issue was covering up this one? As before though, this actually affects all non-Pylance LS features in the notebook context, and would be great to have fixed.

DonJayamanne commented 2 days ago

@karthiknadig this is LSP related, not Jupyter, given jupyter doesn't own LSP for Jedi, moving back to Python extension. If you read the user comments, it works in the first cell and not the second cell, meaning LSP isn't able to determine the fact that second cell is part of the notebook or the like.

We already had some other issues in the past in this same space, https://github.com/microsoft/vscode-python/pull/23898

@tmchartrand Sorry you are running into this,

"middleware addon component"

What changes are those? Please can you try a different version of Jupyter and or Python extension, perhaps we can find out where this went wrong. I'll try this at my end as well.

karthiknadig commented 2 days ago

@DonJayamanne This might be a limitation of jedi-language-server at this point. LSP now has full support for notebooks but the jedi language server implementation still only looks at whole documents. It gets partial view, because it is seeing each cell as a whole document. If jedi-language-server uses the notebook features in LSP, it should be able to provide a richer experience here.

Created a feature request on jedi-language-server: https://github.com/pappasam/jedi-language-server/issues/324

DonJayamanne commented 2 days ago

Weird, I thought this did work in the past with Jedi LSP.

DonJayamanne commented 2 days ago

I can confirm, this is a regression, I can verify that it worked with the older versions

non-Pylance intellisense only works as expected in the notebook for jupyter v2023.9.100, python v2024.2.1 or older. For newer

See here https://github.com/microsoft/vscode-python/issues/23897

DonJayamanne commented 2 days ago

Is it possible some code was removed that was actually crucial for Jedi LSP, e.g. some of the middle ware or the like.

tmchartrand commented 2 days ago

Sounds like you made the connection, but "middleware addon component" was referring to the PR resolving the previous issue, https://github.com/microsoft/vscode-python/pull/23898

karthiknadig commented 2 days ago

@DonJayamanne I tried versions all the way back to 2023.0.* (keeping jupyter version latest), I was not able to get it to work just moving python back. Moving Jupyter back to 2023.9.100 and python to 2024.2.1, it seems to look for Notebook Concatenator. Triggering goto definition on something in second cell causes this: Image

This is what I was trying to mention, previously we used to concatenate cells into a virtual document. I think this got removed because LSP now fully supports notebooks.

Was the concatenator in Jupyter or in Python extension. I can't seem to find it.

DonJayamanne commented 1 day ago

@karthiknadig Yes, I believe this was in Jupyter extension. Let me see what we can do to address this. Lets leave this issue in this repo for now. I think we might want to just create a Jedi LS extension for now, instead of moving stuff back into Jupyter, when this can be fixed by Jedi in the future (when the Notebook LSP protocol parts are implemented).

Will try to whip up an extension for this and see where we can go from there.