enhancement to the auto-exclusion filter to pick up full Python runtime installations in addition to picking up virtual environments.

[ncoghlan]

Environment data

• Language Server version: 2024.6.1
• OS and version: linux x64
• Python version (and distribution if applicable, e.g. Anaconda): see repro details below
• python.analysis.indexing: true
• python.analysis.typeCheckingMode: off

Code Snippet

Code snippet was just from pathlib import Path.

A separate stdlib import in the same file using the import json syntax did not get a warning.

Repro Steps

Initial occurrence:

1. Working on file containing from pathlib import Path using venv A (not the right venv for this file, so many modules are missing). venv A is based on a system level Python 3.11 installation (Fedora 40 Remix for WSL )
2. Switch to venv B using the venv selector in the lower right (the actual venv for this file, with all its dependencies available). venv B is based on a pdm-installed Python 3.11 runtime published by the python-build-standalone project.

Subsequent occurrence:

Starting VSCode with venv B as the active venv is enough to display the problem (no venv switching required).

Additional info:

• both venv A and venv B are in local build folders excluded by .gitignore (and listed as automatically excluded as the language server starts up)
• the PDM-installed python-build-standalone runtime install that venv B uses is also in the same local build folder and NOT listed as automatically excluded when the language server starts up (so Python runtime installs inside the project folder would be another potential candidate for Pylance getting confused about file shadowing if my active venv switching hypothesis is wrong)
• the Python: Restart Language Server command does NOT clear the spurious warning
• the Pylance: Clear All Persisted Indices command followed by the Python: Restart Language Server command does NOT clear the spurious warning
• a full restart of VSCode (and the machine since I shut it down overnight) did NOT clear the spurious warning

I originally assumed that the reported problem was related to switching the active Python virtual environment and something getting confused in a stale cache somewhere, but that now seems unlikely given that clearing caches and restarting VSCode didn't change the behaviour.

Expected behavior

No warning, since the referenced pathlib.py instance is the stdlib module.

Actual behavior

Pylance emits the '"/path/to/python_build_standalone_install/cpython@3.11/lib/python3.11/pathlib.py" is overriding the stdlib module "pathlib"' warning mentioned in the issue title

[rchiodo]

Thanks for the issue. It sounds like our check for the stdlib override is only looking in its own venv to see if the file isn't part of the stdlib. We probably need to check if the path is excluded as well.

[ncoghlan]

I updated my original post to reflect that the problem is still there even after restarting VSCode (and the host machine), so the fact it originally happened after switching venvs now looks like it was just a coincidence.

The cpython@3.11 build subfolder isn't getting listed as automatically excluded when PyLance starts up, though (unlike the various venv folders in the same build directory). Having a full Python install inside a project build folder (vs a venv referencing a Python installed elsewhere) is admittedly pretty strange, so maybe that folder not getting excluded by PyLance actually is the problem? (The Python installation is being created by running pdm python install cpython@3.11 with PDM's python.install_root target path set to the project's build folder).

For a bit of extra headscratching, the file where I first encountered this problem is now loading more stdlib modules and is now inconsistent as to whether PyLance considers them to be shadowed or not:

import secrets gets flagged, import logging doesn't from pathlib ... and from typing ... are both flagged, but a just-to-try-it-out from logging ... import was left unflagged

json and os are imported later in the file, and are also not flagged.

I tried turning on PyLance's logging, but nothing jumped out as being relevant to the reported issue. Please let me know if there's something specific worth looking for.

[bschnurr]

As a work around did you try excluding the folder with python? "/path/to/python_build_standalone_install/cpython@3.11/lib/python3.11/pathlib.py

"python.analysis.exclude": [ "path" ]

[ncoghlan]

I hadn't tried that, but adding explicit exclusions for the build & test folders did indeed make the problem go away.

That suggests if anything is to be changed here, it would be an enhancement to the auto-exclusion filter to pick up full Python runtime installations in addition to picking up virtual environments.

Adding the explicit exclusion also fixed the following in the language server log:

Before: 2024-07-06 14:10:38.162 [info] [Info - 2:10:38 PM] (1576) Found 5087 source files

After: 2024-07-06 14:11:18.593 [info] [Info - 2:11:18 PM] (1576) Found 7 source files

I suspect the intermittent nature of the warning related to the following log entry for the previous folder config where the build & test runtime installations were being picked up: 2024-07-06 14:10:54.223 [info] [Warn - 2:10:54 PM] (1576) Workspace indexing has hit its upper limit: 2000 files

If some of the stdlib files in the build & test folders weren't indexed, those are presumably the ones that PyLance didn't emit a standard lib shadowing warning about.