Presence of empty `pyproject.toml` overrides server configuration

[sventec]

I have basedpyright configured as a server via nvim-lspconfig, and have diagnostic severity overrides in the settings there. After updating basedpyright from 1.12.3 to 1.12.4, it seems that the presence of a pyproject.toml in the project root (even if completely empty) resets my configured diagnostic severity overrides to the defaults.

I believe this could be an issue with basedpyright, as the only thing that I changed to reproduce this issue is the version from 1.12.3 (overrides work) to 1.12.4 (overrides gone if empty pyproject exists).

An example is below, with the following basedpyright configuration (via nvim-lsp, hence the lua-style nesting):

settings = {
  basedpyright = {
    analysis = {
      diagnosticSeverityOverrides = {
        reportAny = false,
      },
    },
  },
},

My project directory only contains:

./test.py
./pyproject.toml

Note that pyproject.toml is completely empty. There are no settings for basedpyright (or anything else).

Here is what test.py looks like with basedpyright 1.12.3 (no diagnostic):

And here is what test.py looks like with basedpyright 1.12.4 (no other changes):

As visible in these screenshots, the override disabling reportAny no longer works on basedpyright 1.12.4, when there is an empty pyproject.toml file in the project root. Removing the pyproject.toml file allows the override to work again, but this is not ideal as many Python projects contain this file.

I would be happy to provide other information as needed, or attempt to reproduce in other environments. As I was able to isolate only the basedpyright version in reproducing this, I figured the report, even including my other neovim configuration, would still be helpful.

[sventec]

My apologies, this looks like a problem with upstream pyright. I was able to reproduce using pyright 1.1.364 to 1.1.365.

I'll try to clean up my report to get it submitted there.

[sventec]

I'm still trying to track down more details on this. I checked neovim's LSP log, and it seems that it's sending the proper internal LSP configuration via a didChangeConfiguration notification, but shortly after is sending another didChangeConfiguration notification with an empty set of params.

My best guess so far is that something changed in how new config notifications are merged within pyright? Maybe this second empty notification would previously extend, and thus keep, the previously sent configuration, but is now overwriting it?

Still trying to wrap my head around the LSP behavior a bit better and track this down.

[willofferfit]

@sventec were you able to track the issue down and/or report it upstream?

[sventec]

@willofferfit I haven't yet reported anything upstream; I was having trouble pinning anything down with neovim, mostly due to my own unfamiliarity with LSP communications. That issue #418 seems more promising though, reproducing in vscode eliminates many of the variables associated with my neovim setup that would likely get the issue dismissed if reported upstream.

I'll see if I can reproduce in a fresh vscode install with pyright 1.1.364 (last working version I found) to latest.

[DetachHead]

just confirmed that it was an empty pyproject.toml causing #418 as well, so i'm pretty sure they're the same issue

[sventec]

I was able to reproduce #418 in a fresh vscode install with a workspace settings.json. Interestingly, I am unable to reproduce if the same settings are included in vscode's user settings.json, only the workspace config seems to cause this.

[willofferfit]

In my case, I'm using vscodium v1.90.2 and BasedPyright extension v1.13.1. I have a non-empty pyproject.toml, which does not contain any pyright configuration (I have an external .json for CI). I have the following keys on User/settings.json:

{
    "python.languageServer": "None",
    "basedpyright.analysis.diagnosticMode": "workspace",
    "basedpyright.analysis.typeCheckingMode": "all",
    "basedpyright.analysis.logLevel": "Error",
    "basedpyright.analysis.diagnosticSeverityOverrides": {
        "reportAny": "warning",
        "reportPrivateUsage": "warning",
        "reportUnknownMemberType": "warning",
        "reportImplicitStringConcatenation": "warning",
    },
}

None of the overrides work

[sventec]

I've raised an upstream issue with the minimal reproduction provided by @DetachHead. See microsoft/pyright#8239

[sventec]

@willofferfit just to confirm, if you downgrade basedpyright to v1.12.3, do the overrides work as expected? That's the behavior I was seeing, any pyright version <= 1.1.364 works as intended.

[willofferfit]

@sventec Forgot to mention that I have import strategy set to fromEnvironment, and I have pyright v1.1.368 installed. Will try to downgrade and report back

[willofferfit]

@sventec I first downgraged pyright in my venv to v1.1.364 and reloaded vscodium – overrides continued to not work. Then I downgraded basedpyright extension to v1.12.3 and overrides started to work as expected. Then upgraded basedpyright to v1.12.5 and overrides stopped working.

How can I know if basedpyright is truly using my venv's pyright? I get the logs below, which shows it's using my venv's python but doesn't say anything about pyright

[Info  - 3:37:52 PM] basedpyright language server 1.12.3 starting
[Info  - 3:37:52 PM] Server root directory: file:///[...]/.vscode-oss/extensions/detachhead.basedpyright-1.12.3-universal/dist
[Info  - 3:37:52 PM] Starting service instance "[...]"
Received pythonPath from Python extension: /[...]/.venv/bin/python
Received pythonPath from Python extension: /[...]/.venv/bin/python
Received pythonPath from Python extension: /[...]/.venv/bin/python

[sventec]

@willofferfit That's interesting, I haven't used fromEnvironment before. I'm not sure that it uses the pyright from your environment, the docs here sound as though it's instead looking in your environment for basedpyright (which uses a bundled pyright version?):

basedpyright fixes this problem by adding an importStrategy option to the extension, which defaults to looking in your project for the basedpyright pypi package.

[sventec]

Issue closed in pyright, there is an upstream issue for this in pylance here: microsoft/pylance-release#6052

[DetachHead]

How can I know if basedpyright is truly using my venv's pyright? I get the logs below, which shows it's using my venv's python but doesn't say anything about pyright

@willofferfit the logging for that shows up in the devtools console instead: Help > Toggle Developer Tools (i should probably have figured out how to log it to the extension output instead, but this was one of the first features i added when i had no idea what i was doing lol). it should say something like "using pyright executable" and the path to the pyright-langserver executable from your environment:

if it fails to find it, it should instead print "failed to find pyright executable, falling back to bundled" (currently the path it outputs in this message is wrong though #197)

---

That's interesting, I haven't used fromEnvironment before. I'm not sure that it uses the pyright from your environment, the docs here sound as though it's instead looking in your environment for basedpyright (which uses a bundled pyright version?)

@sventec "fromEnvironment" means that it's will look inside your currently installed python packages for the basedpyright package. if it can't find it, it falls back to "useBundled", which means that it uses the version of basedpyright that comes bundled with the vscode extension

[willofferfit]

@DetachHead thanks for pointing that out; indeed my install failed to find venv's pyright:

[Extension Host] failed to find pyright executable, falling back to bundled: /[...]/.venv/bin/basedpyright-langserver
[Extension Host] using bundled pyright

Thing is, inside .venv/bin I don't have basedpyright-langserver, which is mentioned, but I have these:

How should I debug it? Also, since this is offtopic, do you want me to open another issue to talk about it?

[DetachHead]

are you using pyright instead of basedpyright? unfortunately the basedpyright vscode extension does not work with the pyright pypi package (pyright-langserver), which i'm guessing is what you're using. it will only work with the basedpyright package (basedpyright-langserver).

i originally intended to support using the basedpyright vscode extension with regular pyright, but there was an issue that caused the language server to not work at all, which i could only fix by commenting out this part of the webpack config for some reason. so i decided that if i was going to have to patch the language server anyway, i wasn't going to bother trying to support both.

you can make basedpyright behave pretty much exactly the same as pyright by setting typeCheckingMode to "standard" anyway.

feel free to raise another issue if you want us to revisit this

[willofferfit]

oh makes sense! indeed I use pypi's pyright, and basedpyright only as a vscodium extension (at least for now). thanks @DetachHead

[sventec]

Reported closed by pyright commit https://github.com/microsoft/pyright/commit/5d5fe5d15c90808a1d60d3969b5b72fa0daea6b6.

Should be available in next upcoming pyright version. Leaving it up to maintainers as to whether this issue should be closed now, or remain open until the fix has been merged into basedpyright.

[DetachHead]

will leave this open until i merge the next upstream version