search

astral-sh / ruff-vscode

A Visual Studio Code extension with support for the Ruff linter.
Other
1.04k stars 52 forks source link

Native server does not work with Jupyter Notebook without configuration #546

Closed Xiao-Chenguang closed 3 weeks ago

Xiao-Chenguang commented 1 month ago

Ruff not function in jupyter notebooks when set ruff.nativeServer to auto or on. It seems there are some communication problem between the nativeserver and notebook. When use ruff-lsp by setting ruff.nativeServer to off, all ruff functions works as expected.

According to the server log, the nativeserver started normally but receive nothing while I type format command.

2024-07-21 16:44:25.149 [info] Name: Ruff
2024-07-21 16:44:25.149 [info] Module: ruff
2024-07-21 16:44:25.149 [info] Python extension loading
2024-07-21 16:44:25.149 [info] Waiting for interpreter from python extension.
2024-07-21 16:44:25.149 [info] Python extension loaded
2024-07-21 16:44:25.242 [info] Falling back to bundled executable: /Users/cg/.vscode/extensions/charliermarsh.ruff-2024.34.0-darwin-arm64/bundled/libs/bin/ruff
2024-07-21 16:44:25.265 [info] Resolved 'ruff.nativeServer: auto' to use the native server
2024-07-21 16:44:25.266 [info] Found Ruff 0.5.3 at /Users/cg/.vscode/extensions/charliermarsh.ruff-2024.34.0-darwin-arm64/bundled/libs/bin/ruff
2024-07-21 16:44:25.266 [info] Server run command: /Users/cg/.vscode/extensions/charliermarsh.ruff-2024.34.0-darwin-arm64/bundled/libs/bin/ruff server
2024-07-21 16:44:25.266 [info] Server: Start requested.

The behaviour of setting nativeServer off is:

2024-07-21 16:48:31.470 [info] Name: Ruff
2024-07-21 16:48:31.470 [info] Module: ruff
2024-07-21 16:48:31.470 [info] Python extension loading
2024-07-21 16:48:31.470 [info] Waiting for interpreter from python extension.
2024-07-21 16:48:31.470 [info] Python extension loaded
2024-07-21 16:48:31.470 [info] Server run command: /Users/cg/miniconda3/envs/fl/bin/python /Users/cg/.vscode/extensions/charliermarsh.ruff-2024.34.0-darwin-arm64/bundled/tool/server.py
2024-07-21 16:48:31.470 [info] Server: Start requested.
2024-07-21 16:48:31.934 [info] 2024-07-21 16:48:31,934 INFO Starting IO server

2024-07-21 16:48:31.975 [info] Workspace settings: []
2024-07-21 16:48:31.975 [info] Global settings: {
    "nativeServer": "off",
    "cwd": "/",
    "workspace": "/",
    "path": [],
    "ignoreStandardLibrary": true,
    "interpreter": [],
    "configuration": null,
    "importStrategy": "fromEnvironment",
    "codeAction": {
        "fixViolation": {
            "enable": true
        },
        "disableRuleComment": {
            "enable": true
        }
    },
    "lint": {
        "enable": true,
        "run": "onType",
        "args": []
    },
    "format": {
        "args": []
    },
    "enable": true,
    "organizeImports": true,
    "fixAll": true,
    "showNotifications": "off",
    "configurationPreference": "editorFirst",
    "showSyntaxErrors": true
}
2024-07-21 16:48:31.990 [info] Interpreter executable (/Users/cg/miniconda3/envs/fl/bin/ruff) not found
2024-07-21 16:48:31.990 [info] Falling back to bundled executable: /Users/cg/.vscode/extensions/charliermarsh.ruff-2024.34.0-darwin-arm64/bundled/libs/bin/ruff
2024-07-21 16:48:32.025 [info] Inferred version 0.5.3 for: /Users/cg/.vscode/extensions/charliermarsh.ruff-2024.34.0-darwin-arm64/bundled/libs/bin/ruff
2024-07-21 16:48:32.025 [info] Found ruff 0.5.3 at /Users/cg/.vscode/extensions/charliermarsh.ruff-2024.34.0-darwin-arm64/bundled/libs/bin/ruff
2024-07-21 16:48:32.025 [info] Running Ruff with: /Users/cg/.vscode/extensions/charliermarsh.ruff-2024.34.0-darwin-arm64/bundled/libs/bin/ruff ['check', '--force-exclude', '--no-cache', '--no-fix', '--quiet', '--output-format', 'json', '-', '--stdin-filename', '/Users/cg/Desktop/test.ipynb']

other ruff setting: all default ruff version: v2024.34.0 OS: Mac Python version: 3.10

Mininal example:

# test.ipynb
a = [[1, 2, 3, 4, 5], [6, 7, 8, 9, 10], [11, 12, 13, 14, 15], [16, 17, 18, 19, 20], [21, 22, 23, 24, 25]]
charliermarsh commented 1 month ago

\cc @dhruvmanila

dhruvmanila commented 1 month ago

Can you provide the contents of the Ruff config in pyproject.toml / ruff.toml / .ruff.toml file? After https://github.com/astral-sh/ruff/pull/12252, Ruff would only include the notebook files if it's included via either include or extend-include.

The reason it works for ruff-lsp is because it uses the ruff CLI directly via stdin and it would always provide diagnostics / formatting for a file which is passed directly:

2024-07-22 10:08:25.812 [info] Running Ruff with: /Users/dhruv/work/astral/ruff/target/debug/ruff ['check', '--force-exclude', '--no-cache', '--no-fix', '--quiet', '--output-format', 'json', '-', '--stdin-filename', '/Users/dhruv/playground/ruff/notebooks/test.ipynb']

This doesn't consider the include / extend-include config because the --force-exclude flag doesn't consider those settings.

dhruvmanila commented 1 month ago

You can also enabled debug logs for the native server using:

{
  "ruff.trace.server": "messages",
  "ruff.logLevel": "debug"
}

which outputs:

2024-07-22 10:03:54.657 [info] Using 'path' setting: /Users/dhruv/work/astral/ruff/target/debug/ruff
2024-07-22 10:03:55.471 [info] Found Ruff 0.5.2 at /Users/dhruv/work/astral/ruff/target/debug/ruff
2024-07-22 10:03:55.471 [info] Server run command: /Users/dhruv/work/astral/ruff/target/debug/ruff server --preview
2024-07-22 10:03:55.472 [info] Server: Start requested.
2024-07-22 10:03:55.476 [info] [Trace - 10:03:55 AM] Sending request 'initialize - (0)'.
2024-07-22 10:03:55.488 [info] [Trace - 10:03:55 AM] Received response 'initialize - (0)' in 12ms.
2024-07-22 10:03:55.489 [info] [Trace - 10:03:55 AM] Sending notification 'initialized'.
2024-07-22 10:03:55.490 [info] [Trace - 10:03:55 AM] Sending notification 'textDocument/didOpen'.
2024-07-22 10:03:55.490 [info] [Trace - 10:03:55 AM] Sending request 'textDocument/diagnostic - (1)'.
2024-07-22 10:03:55.490 [info] [Trace - 10:03:55 AM] Sending notification 'notebookDocument/didOpen'.
2024-07-22 10:03:55.492 [info] [Trace - 10:03:55 AM] Sending request 'textDocument/codeAction - (2)'.
2024-07-22 10:03:55.493 [info] [Trace - 10:03:55 AM] Sending request 'textDocument/codeAction - (3)'.
2024-07-22 10:03:55.494 [info] [Trace - 10:03:55 AM] Sending request 'textDocument/codeAction - (4)'.
2024-07-22 10:03:55.510 [info]    0.019833583s DEBUG ThreadId(13) ruff_server::session::index::ruff_settings: Ignored path via `exclude`: /Users/dhruv/playground/ruff/.vscode

2024-07-22 10:03:55.519 [info] [Trace - 10:03:55 AM] Received request 'client/registerCapability - (1)'.
2024-07-22 10:03:55.519 [info] [Trace - 10:03:55 AM] Sending response 'client/registerCapability - (1)'. Processing request took 1ms
2024-07-22 10:03:55.519 [info]    0.028470583s DEBUG ruff:worker:0 ruff_server::resolve: Included path via `include`: /Users/dhruv/playground/ruff/src/play.py

2024-07-22 10:03:55.520 [info]    0.029595292s  INFO     ruff:main ruff_server::server: Configuration file watcher successfully registered

2024-07-22 10:03:55.520 [info] [Trace - 10:03:55 AM] Received response 'textDocument/codeAction - (4)' in 26ms.

You can see that it only included the Python file (search for "Included path via") and not the notebook file.

But, once you include it in the config:

[tool.ruff]
extend-include = ["*.ipynb"]

It'll start including the notebook files as well.

Xiao-Chenguang commented 1 month ago

You can also enabled debug logs for the native server using:

{
  "ruff.trace.server": "messages",
  "ruff.logLevel": "debug"
}

which outputs:

2024-07-22 10:03:54.657 [info] Using 'path' setting: /Users/dhruv/work/astral/ruff/target/debug/ruff
2024-07-22 10:03:55.471 [info] Found Ruff 0.5.2 at /Users/dhruv/work/astral/ruff/target/debug/ruff
2024-07-22 10:03:55.471 [info] Server run command: /Users/dhruv/work/astral/ruff/target/debug/ruff server --preview
2024-07-22 10:03:55.472 [info] Server: Start requested.
2024-07-22 10:03:55.476 [info] [Trace - 10:03:55 AM] Sending request 'initialize - (0)'.
2024-07-22 10:03:55.488 [info] [Trace - 10:03:55 AM] Received response 'initialize - (0)' in 12ms.
2024-07-22 10:03:55.489 [info] [Trace - 10:03:55 AM] Sending notification 'initialized'.
2024-07-22 10:03:55.490 [info] [Trace - 10:03:55 AM] Sending notification 'textDocument/didOpen'.
2024-07-22 10:03:55.490 [info] [Trace - 10:03:55 AM] Sending request 'textDocument/diagnostic - (1)'.
2024-07-22 10:03:55.490 [info] [Trace - 10:03:55 AM] Sending notification 'notebookDocument/didOpen'.
2024-07-22 10:03:55.492 [info] [Trace - 10:03:55 AM] Sending request 'textDocument/codeAction - (2)'.
2024-07-22 10:03:55.493 [info] [Trace - 10:03:55 AM] Sending request 'textDocument/codeAction - (3)'.
2024-07-22 10:03:55.494 [info] [Trace - 10:03:55 AM] Sending request 'textDocument/codeAction - (4)'.
2024-07-22 10:03:55.510 [info]    0.019833583s DEBUG ThreadId(13) ruff_server::session::index::ruff_settings: Ignored path via `exclude`: /Users/dhruv/playground/ruff/.vscode

2024-07-22 10:03:55.519 [info] [Trace - 10:03:55 AM] Received request 'client/registerCapability - (1)'.
2024-07-22 10:03:55.519 [info] [Trace - 10:03:55 AM] Sending response 'client/registerCapability - (1)'. Processing request took 1ms
2024-07-22 10:03:55.519 [info]    0.028470583s DEBUG ruff:worker:0 ruff_server::resolve: Included path via `include`: /Users/dhruv/playground/ruff/src/play.py

2024-07-22 10:03:55.520 [info]    0.029595292s  INFO     ruff:main ruff_server::server: Configuration file watcher successfully registered

2024-07-22 10:03:55.520 [info] [Trace - 10:03:55 AM] Received response 'textDocument/codeAction - (4)' in 26ms.

You can see that it only included the Python file (search for "Included path via") and not the notebook file.

But, once you include it in the config:

[tool.ruff]
extend-include = ["*.ipynb"]

It'll start including the notebook files as well.

Thank you @dhruvmanila , setting extend-include = ["*.ipynb"] in a ruff.toml file works for me. I didn't use ruff config file before and I would suggest you put this as a notice at some place obvious.

juanitorduz commented 1 month ago

Thanks for the tip! I was having the same problem and 

[tool.ruff]
extend-include = ["*.ipynb"]

solved it for me as well!

dhruvmanila commented 1 month ago

No worries. It makes sense for this change to be documented. I'm thinking of adding it to:

  1. https://github.com/astral-sh/ruff-vscode#jupyter-notebook-support
  2. https://docs.astral.sh/ruff/editors/features/#jupyter-notebook

Do you have any other places in mind which will be more visible?

Xiao-Chenguang commented 1 month ago

That's really helpful. Use it as an extension, I would first check the extension readme as you mentioned and then the extension setting. It may benefit users by adding this into extension setting and the notebook feature can be enabled global without need of toml file per project.

dhruvmanila commented 1 month ago

It may benefit users by adding this into extension setting and the notebook feature can be enabled global without need of toml file per project.

Yeah, I had a similar thought earlier but that would mean that running Ruff from the command-line would give different results compared to the editor. I'm not sure if there are any opinions on this but as the notebook files were always included for linting and formatting via ruff-lsp and there haven't been any user report, I guess that should be fine (?).

Xiao-Chenguang commented 1 month ago

I'm not sure if there are any opinions on this but as the notebook files were always included for linting and formatting via ruff-lsp and there haven't been any user report, I guess that should be fine (?).

Totally agree. Just keep the notebook behaviour similar to ruff-lsp before there is a better solution.

dhruvmanila commented 1 month ago

We might just include Jupyter Notebooks in Ruff's inclusion by default (https://github.com/astral-sh/ruff/issues/12456).

rnckp commented 1 month ago

@dhruvmanila

I still have difficulties making it work, even when I add extend-include = ["*.ipynb"] in ~.vscode/extensions/charliermarsh.ruff-2024.36.0-darwin-arm64/pyproject.toml.

.py files in VS Code are being formatted and linted properly. However, jupyer notebooks are not.

Tried to restart, reinstall from clean state (VS Code, settings and extensions) to no avail. Any recommendations to make it work again? I noticed that I stopped working a couple of days ago.

I use all latest versions of VS Code and extensions. Besides Ruff I have installed Python, Pylance, Python Debugger (all MS), Jupyter, Jupyter Keymap and Jupyter Notebook Renderers.

dhruvmanila commented 1 month ago

I still have difficulties making it work, even when I add extend-include = ["*.ipynb"] in ~.vscode/extensions/charliermarsh.ruff-2024.36.0-darwin-arm64/pyproject.toml.

I think you'd need to add that in the pyproject.toml file within the current project you're in. That directory is where the extension resides. For example, if you opened VS Code in ~/projects/ruff, then the root config file would be ~/projects/ruff/pyproject.toml.

rnckp commented 1 month ago

That did the trick! Thanks so much! You saved my day... 🙏☀️

A detail: In my case it was sufficient to create pyproject.toml just in the root folder of my project (aka the top code folder that I opened in my VS Code workspace). Subfolder /ruff/ didn't seem to be necessary.

dhruvmanila commented 1 month ago

~/projects/ruff is the root folder of the "ruff" project ;)

rnckp commented 1 month ago

Got it! Thanks! 😊

DGrady commented 1 month ago

We might just include Jupyter Notebooks in Ruff's inclusion by default (astral-sh/ruff#12456).

Upvoting this! Speaking as someone who mainly uses Ruff from within Jupyter notebooks running in VSCode, my experience here was: a few months ago, very happy that I got to replace two slow extensions with one fast one; then recent frustration when the fast one stopped working with no obvious errors.