PyLance not recognizing imports from PEP-660 editable installs

rzats commented 1 year ago

Environment data

Language Server version: 2022.10.20
OS and version: darwin x64
Python version (and distribution if applicable, e.g. Anaconda): 3.8.15
python.analysis.indexing: false
python.analysis.typeCheckingMode: off

Code Snippet

I'm currently trying to get static analysis working for a project with a nonstandard directory structure (more at https://github.com/microsoft/pylance-release/issues/3454). Turning it into an editable install seem like the best option so far.

https://github.com/rzats/setuptools-repro is a small repo that mirrors the structure of that project. It contains this pyproject.toml:

[build-system]
requires = ["setuptools>=65", "wheel"]
build-backend = "setuptools.build_meta"

setup.cfg:

[metadata]
name = Local Package

[options]
packages =
    root.package

package_dir =
    root.package = root/package/src

as well as a Python file at root/package/src/test.py, and a main.py that tries to import test.py.

Repro Steps

git clone https://github.com/rzats/setuptools-repro.git
cd setuptools-repro
python -m venv .venv
.venv/bin/python -m pip install -U pip
.venv/bin/python -m pip install -e .
.venv/bin/python -c 'import root.package.test; print(root.package.test)'
<module 'root.package.test' from '/Users/work/Documents/setuptools-repro/root/package/src/test.py'>

Expected behavior

root.package.test should show up as a valid import in VSCode too.

Actual behavior

That doesn't happen:

screenshot of source code inside main.py that reads "import root.package.test as test" overlaid by an error message that reads "Import "root.package.test" could not be resolved (reportMissingImports)"

Unfortunately, reverting to legacy editable packages as suggested by https://github.com/microsoft/pylance-release/issues/3265#issuecomment-1240306780 does not work for me, since that has an unrelated issue where advanced package_dir bindings don't work at all - the newer PEP-660 implementation does support that.

Logs

Python

``` LSP Notebooks experiment is enabled LSP Notebooks interactive window support is enabled Python interpreter path: ./.venv/bin/python > conda info --json > ./.venv/bin/python ~/.vscode/extensions/ms-python.python-2022.16.0/pythonFiles/get_output_via_markers.py ~/.vscode/extensions/ms-python.python-2022.16.0/pythonFiles/interpreterInfo.py > /usr/bin/python3 ~/.vscode/extensions/ms-python.python-2022.16.0/pythonFiles/get_output_via_markers.py ~/.vscode/extensions/ms-python.python-2022.16.0/pythonFiles/interpreterInfo.py > /usr/local/bin/python3.8 ~/.vscode/extensions/ms-python.python-2022.16.0/pythonFiles/get_output_via_markers.py ~/.vscode/extensions/ms-python.python-2022.16.0/pythonFiles/interpreterInfo.py > . ./.venv/bin/activate && echo 'e8b39361-0157-4923-80e1-22d70d46dee6' && python ~/.vscode/extensions/ms-python.python-2022.16.0/pythonFiles/printEnvVariables.py Starting Pylance language server. ```

Python Language Server

``` [Info - 6:15:09 PM] (17998) Pylance language server 2022.10.20 (pyright e2d13ab1) starting [Info - 6:15:09 PM] (17998) Server root directory: /Users/work/.vscode/extensions/ms-python.vscode-pylance-2022.10.20/dist [Info - 6:15:09 PM] (17998) Starting service instance "setuptools-repro" [Info - 6:15:09 PM] (17998) Notebook support: LSP [Info - 6:15:09 PM] (17998) Interactive window support: LSP [Info - 6:15:09 PM] (17998) No configuration file found. [Info - 6:15:09 PM] (17998) pyproject.toml file found at /Users/work/Documents/setuptools-repro. [Info - 6:15:09 PM] (17998) Setting pythonPath for service "setuptools-repro": "/Users/work/Documents/setuptools-repro/.venv/bin/python" [Info - 6:15:09 PM] (17998) Loading pyproject.toml file at /Users/work/Documents/setuptools-repro/pyproject.toml [Error - 6:15:09 PM] (17998) Pyproject file "/Users/work/Documents/setuptools-repro/pyproject.toml" is missing "[tool.pyright]" section. [Warn - 6:15:09 PM] (17998) stubPath /Users/work/Documents/setuptools-repro/typings is not a valid directory. [Info - 6:15:09 PM] (17998) Assuming Python version 3.8 [Info - 6:15:09 PM] (17998) Assuming Python platform Darwin [Info - 6:15:09 PM] (17998) Search paths for /Users/work/Documents/setuptools-repro [Info - 6:15:09 PM] (17998) /Users/work/.vscode/extensions/ms-python.vscode-pylance-2022.10.20/dist/typeshed-fallback/stdlib [Info - 6:15:09 PM] (17998) /Users/work/Documents/setuptools-repro [Info - 6:15:09 PM] (17998) /Users/work/Documents/setuptools-repro/typings [Info - 6:15:09 PM] (17998) /Users/work/.vscode/extensions/ms-python.vscode-pylance-2022.10.20/dist/typeshed-fallback/stubs/... [Info - 6:15:09 PM] (17998) /Users/work/.vscode/extensions/ms-python.vscode-pylance-2022.10.20/dist/bundled/stubs [Info - 6:15:09 PM] (17998) /usr/local/Cellar/python@3.8/3.8.15/Frameworks/Python.framework/Versions/3.8/lib/python3.8 [Info - 6:15:09 PM] (17998) /usr/local/Cellar/python@3.8/3.8.15/Frameworks/Python.framework/Versions/3.8/lib/python3.8/lib-dynload [Info - 6:15:09 PM] (17998) /Users/work/Documents/setuptools-repro/.venv/lib/python3.8/site-packages [Info - 6:15:09 PM] (17998) Adding fs watcher for library directories: /usr/local/Cellar/python@3.8/3.8.15/Frameworks/Python.framework/Versions/3.8/lib/python3.8 /usr/local/Cellar/python@3.8/3.8.15/Frameworks/Python.framework/Versions/3.8/lib/python3.8/lib-dynload /Users/work/Documents/setuptools-repro/.venv/lib/python3.8/site-packages [Info - 6:15:09 PM] (17998) Adding fs watcher for directories: /Users/work/Documents/setuptools-repro [Info - 6:15:09 PM] (17998) Searching for source files [Info - 6:15:09 PM] (17998) Found 2 source files (17998) [IDX(FG)] index libraries /Users/work/Documents/setuptools-repro (index) ... (17998) [IDX(FG)] read stdlib indices (27ms) (17998) [IDX(FG)] index libraries /Users/work/Documents/setuptools-repro (index) [succeed] (29ms) (17998) [FG] parsing: /Users/work/Documents/setuptools-repro/main.py (13ms) [Info - 6:15:09 PM] (17998) Could not import 'root.package.test' in file '/Users/work/Documents/setuptools-repro/main.py' [Info - 6:15:09 PM] (17998) Looking in stubPath '/Users/work/Documents/setuptools-repro/typings' [Info - 6:15:09 PM] (17998) Attempting to resolve stub package using root path '/Users/work/Documents/setuptools-repro/typings' [Info - 6:15:09 PM] (17998) Attempting to resolve using root path '/Users/work/Documents/setuptools-repro/typings' [Info - 6:15:09 PM] (17998) Looking in root directory of execution environment '/Users/work/Documents/setuptools-repro' [Info - 6:15:09 PM] (17998) Attempting to resolve stub package using root path '/Users/work/Documents/setuptools-repro' [Info - 6:15:09 PM] (17998) Attempting to resolve using root path '/Users/work/Documents/setuptools-repro' [Info - 6:15:09 PM] (17998) Looking in python search path '/usr/local/Cellar/python@3.8/3.8.15/Frameworks/Python.framework/Versions/3.8/lib/python3.8' [Info - 6:15:09 PM] (17998) Attempting to resolve stub package using root path '/usr/local/Cellar/python@3.8/3.8.15/Frameworks/Python.framework/Versions/3.8/lib/python3.8' [Info - 6:15:09 PM] (17998) Attempting to resolve using root path '/usr/local/Cellar/python@3.8/3.8.15/Frameworks/Python.framework/Versions/3.8/lib/python3.8' [Info - 6:15:09 PM] (17998) Looking in python search path '/usr/local/Cellar/python@3.8/3.8.15/Frameworks/Python.framework/Versions/3.8/lib/python3.8/lib-dynload' [Info - 6:15:09 PM] (17998) Attempting to resolve stub package using root path '/usr/local/Cellar/python@3.8/3.8.15/Frameworks/Python.framework/Versions/3.8/lib/python3.8/lib-dynload' [Info - 6:15:09 PM] (17998) Attempting to resolve using root path '/usr/local/Cellar/python@3.8/3.8.15/Frameworks/Python.framework/Versions/3.8/lib/python3.8/lib-dynload' [Info - 6:15:09 PM] (17998) Looking in python search path '/Users/work/Documents/setuptools-repro/.venv/lib/python3.8/site-packages' [Info - 6:15:09 PM] (17998) Attempting to resolve stub package using root path '/Users/work/Documents/setuptools-repro/.venv/lib/python3.8/site-packages' [Info - 6:15:09 PM] (17998) Attempting to resolve using root path '/Users/work/Documents/setuptools-repro/.venv/lib/python3.8/site-packages' [Info - 6:15:09 PM] (17998) Looking in bundled stubs path '/Users/work/.vscode/extensions/ms-python.vscode-pylance-2022.10.20/dist/bundled/stubs' [Info - 6:15:09 PM] (17998) Attempting to resolve stub package using root path '/Users/work/.vscode/extensions/ms-python.vscode-pylance-2022.10.20/dist/bundled/stubs' [Info - 6:15:09 PM] (17998) Attempting to resolve using root path '/Users/work/.vscode/extensions/ms-python.vscode-pylance-2022.10.20/dist/bundled/stubs' [Info - 6:15:09 PM] (17998) Looking for typeshed stdlib path [Info - 6:15:09 PM] (17998) Looking for typeshed stdlib path [Info - 6:15:09 PM] (17998) Attempting to resolve using root path '/Users/work/.vscode/extensions/ms-python.vscode-pylance-2022.10.20/dist/typeshed-fallback/stdlib' [Info - 6:15:09 PM] (17998) Typeshed path not found [Info - 6:15:09 PM] (17998) Looking for typeshed third-party path [Info - 6:15:09 PM] (17998) Looking for typeshed stubs path [Info - 6:15:09 PM] (17998) Typeshed path not found (17998) [FG] parsing: /Users/work/.vscode/extensions/ms-python.vscode-pylance-2022.10.20/dist/typeshed-fallback/stdlib/builtins.pyi [fs read 3ms] (70ms) (17998) [FG] binding: /Users/work/.vscode/extensions/ms-python.vscode-pylance-2022.10.20/dist/typeshed-fallback/stdlib/builtins.pyi (24ms) (17998) [FG] binding: /Users/work/Documents/setuptools-repro/main.py (2ms) [Info - 6:15:10 PM] (17998) Background analysis(1) root directory: /Users/work/.vscode/extensions/ms-python.vscode-pylance-2022.10.20/dist [Info - 6:15:10 PM] (17998) Background analysis(1) started (17998) Background analysis message: setConfigOptions (17998) Background analysis message: setImportResolver (17998) Background analysis message: ensurePartialStubPackages (17998) Background analysis message: setTrackedFiles (17998) Background analysis message: markAllFilesDirty (17998) Background analysis message: setFileOpened (17998) Background analysis message: getDiagnosticsForRange (17998) Background analysis message: getSemanticTokens full (17998) [BG(1)] getSemanticTokens full at /Users/work/Documents/setuptools-repro/main.py ... (17998) [BG(1)] parsing: /Users/work/Documents/setuptools-repro/main.py (12ms) [Info - 6:15:10 PM] (17998) Could not import 'root.package.test' in file '/Users/work/Documents/setuptools-repro/main.py' [Info - 6:15:10 PM] (17998) Looking in stubPath '/Users/work/Documents/setuptools-repro/typings' [Info - 6:15:10 PM] (17998) Attempting to resolve stub package using root path '/Users/work/Documents/setuptools-repro/typings' [Info - 6:15:10 PM] (17998) Attempting to resolve using root path '/Users/work/Documents/setuptools-repro/typings' [Info - 6:15:10 PM] (17998) Looking in root directory of execution environment '/Users/work/Documents/setuptools-repro' [Info - 6:15:10 PM] (17998) Attempting to resolve stub package using root path '/Users/work/Documents/setuptools-repro' [Info - 6:15:10 PM] (17998) Attempting to resolve using root path '/Users/work/Documents/setuptools-repro' [Info - 6:15:10 PM] (17998) Looking in python search path '/usr/local/Cellar/python@3.8/3.8.15/Frameworks/Python.framework/Versions/3.8/lib/python3.8' [Info - 6:15:10 PM] (17998) Attempting to resolve stub package using root path '/usr/local/Cellar/python@3.8/3.8.15/Frameworks/Python.framework/Versions/3.8/lib/python3.8' [Info - 6:15:10 PM] (17998) Attempting to resolve using root path '/usr/local/Cellar/python@3.8/3.8.15/Frameworks/Python.framework/Versions/3.8/lib/python3.8' [Info - 6:15:10 PM] (17998) Looking in python search path '/usr/local/Cellar/python@3.8/3.8.15/Frameworks/Python.framework/Versions/3.8/lib/python3.8/lib-dynload' [Info - 6:15:10 PM] (17998) Attempting to resolve stub package using root path '/usr/local/Cellar/python@3.8/3.8.15/Frameworks/Python.framework/Versions/3.8/lib/python3.8/lib-dynload' [Info - 6:15:10 PM] (17998) Attempting to resolve using root path '/usr/local/Cellar/python@3.8/3.8.15/Frameworks/Python.framework/Versions/3.8/lib/python3.8/lib-dynload' [Info - 6:15:10 PM] (17998) Looking in python search path '/Users/work/Documents/setuptools-repro/.venv/lib/python3.8/site-packages' [Info - 6:15:10 PM] (17998) Attempting to resolve stub package using root path '/Users/work/Documents/setuptools-repro/.venv/lib/python3.8/site-packages' [Info - 6:15:10 PM] (17998) Attempting to resolve using root path '/Users/work/Documents/setuptools-repro/.venv/lib/python3.8/site-packages' [Info - 6:15:10 PM] (17998) Looking in bundled stubs path '/Users/work/.vscode/extensions/ms-python.vscode-pylance-2022.10.20/dist/bundled/stubs' [Info - 6:15:10 PM] (17998) Attempting to resolve stub package using root path '/Users/work/.vscode/extensions/ms-python.vscode-pylance-2022.10.20/dist/bundled/stubs' [Info - 6:15:10 PM] (17998) Attempting to resolve using root path '/Users/work/.vscode/extensions/ms-python.vscode-pylance-2022.10.20/dist/bundled/stubs' [Info - 6:15:10 PM] (17998) Looking for typeshed stdlib path [Info - 6:15:10 PM] (17998) Looking for typeshed stdlib path [Info - 6:15:10 PM] (17998) Attempting to resolve using root path '/Users/work/.vscode/extensions/ms-python.vscode-pylance-2022.10.20/dist/typeshed-fallback/stdlib' [Info - 6:15:10 PM] (17998) Typeshed path not found [Info - 6:15:10 PM] (17998) Looking for typeshed third-party path [Info - 6:15:10 PM] (17998) Looking for typeshed stubs path [Info - 6:15:10 PM] (17998) Typeshed path not found (17998) [BG(1)] parsing: /Users/work/.vscode/extensions/ms-python.vscode-pylance-2022.10.20/dist/typeshed-fallback/stdlib/builtins.pyi [fs read 2ms] (79ms) (17998) [BG(1)] binding: /Users/work/.vscode/extensions/ms-python.vscode-pylance-2022.10.20/dist/typeshed-fallback/stdlib/builtins.pyi (29ms) (17998) [BG(1)] binding: /Users/work/Documents/setuptools-repro/main.py (2ms) (17998) [BG(1)] getSemanticTokens full at /Users/work/Documents/setuptools-repro/main.py (126ms) (17998) Background analysis message: getDiagnosticsForRange (17998) Background analysis message: getSemanticTokens range (17998) [BG(1)] getSemanticTokens range 0:0 - 0:32 at /Users/work/Documents/setuptools-repro/main.py (1ms) (17998) Background analysis message: analyze (17998) [BG(1)] analyzing: /Users/work/Documents/setuptools-repro/main.py ... (17998) [BG(1)] checking: /Users/work/Documents/setuptools-repro/main.py ... (17998) [BG(1)] parsing: /Users/work/.vscode/extensions/ms-python.vscode-pylance-2022.10.20/dist/typeshed-fallback/stdlib/typing.pyi [fs read 0ms] (19ms) (17998) [BG(1)] binding: /Users/work/.vscode/extensions/ms-python.vscode-pylance-2022.10.20/dist/typeshed-fallback/stdlib/typing.pyi (5ms) (17998) [BG(1)] parsing: /Users/work/.vscode/extensions/ms-python.vscode-pylance-2022.10.20/dist/typeshed-fallback/stdlib/typing_extensions.pyi [fs read 1ms] (3ms) (17998) [BG(1)] binding: /Users/work/.vscode/extensions/ms-python.vscode-pylance-2022.10.20/dist/typeshed-fallback/stdlib/typing_extensions.pyi (2ms) (17998) [BG(1)] parsing: /Users/work/.vscode/extensions/ms-python.vscode-pylance-2022.10.20/dist/typeshed-fallback/stdlib/_typeshed/__init__.pyi [fs read 0ms] (7ms) (17998) [BG(1)] binding: /Users/work/.vscode/extensions/ms-python.vscode-pylance-2022.10.20/dist/typeshed-fallback/stdlib/_typeshed/__init__.pyi (2ms) (17998) [BG(1)] parsing: /Users/work/.vscode/extensions/ms-python.vscode-pylance-2022.10.20/dist/typeshed-fallback/stdlib/abc.pyi [fs read 0ms] (1ms) (17998) [BG(1)] binding: /Users/work/.vscode/extensions/ms-python.vscode-pylance-2022.10.20/dist/typeshed-fallback/stdlib/abc.pyi (1ms) (17998) [BG(1)] checking: /Users/work/Documents/setuptools-repro/main.py (73ms) (17998) [BG(1)] analyzing: /Users/work/Documents/setuptools-repro/main.py (73ms) (17998) Background analysis message: resumeAnalysis (17998) Background analysis message: getDiagnosticsForRange (17998) Background analysis message: getDiagnosticsForRange (17998) Background analysis message: getDiagnosticsForRange (17998) Background analysis message: getDiagnosticsForRange ```

debonte commented 1 year ago

reverting to legacy editable packages as suggested by https://github.com/microsoft/pylance-release/issues/3265#issuecomment-1240306780 does not work for me

Can you try this instead of SETUPTOOLS_ENABLE_FEATURES="legacy-editable"?:

pip install -e . --config-settings editable_mode=compat

Also, for context, see our docs on PEP 660 editable installs.

rzats commented 1 year ago

Perfect, ~~editable_mode=compat~~ editable_mode=strict resolved the issue! Feel free to close.

debonte commented 1 year ago

@rzats, that's good to know, because setuptools is planning to remove compat mode in the near future and I was under the impression until today that it was equivalent to SETUPTOOLS_ENABLE_FEATURES="legacy-editable".

I'll open an issue with setuptools asking them to keep compat mode around longer, ideally permanently. They indicated that they would be willing to do that if there was a strong argument for it.

I'll keep this issue open until I file the issue with setuptools and then provide a link here.

rzats commented 1 year ago

fwiw, we ended up selecting a slightly different mode, pip install -e . --config-settings editable_mode=strict - this seems to be an implementation exclusive to the setuptools backend that creates symlinks to the working directory in the build subfolder.

debonte commented 1 year ago

Ok, but to be clear, both compat mode and strict mode solve your problem?

rzats commented 1 year ago

Just strict mode - sorry, that's my oversight. I assumed it was identical to compat mode, but in fact it's a separate option that falls back to the non-PEP-660 implementation; and this runs into package_dir issue I've mentioned.

jborean93 commented 1 year ago

Can I ask why this issue was closed, not supporting the default PEP-660 editable install mode is going to confuse a lot of users who can't find this page. Is there a fundamental reason why pylance can't use these libraries when installed in the default mode.

debonte commented 1 year ago

Can I ask why this issue was closed, not supporting the default PEP-660 editable install mode is going to confuse a lot of users who can't find this page. Is there a fundamental reason why pylance can't use these libraries when installed in the default mode.

Unfortunately PEP 660 was approved without discussion with the Python typing community. We cannot run import hooks. But there has been discussion about alternative ways to solve this problem. See this typing-sig thread. If a solution is found, agreed upon by typing-sig, and standardized, Pylance will consider supporting it.

There's more about this in our docs on PEP 660 editable installs.

jborean93 commented 1 year ago

Thanks for the further info, hopefully a decent resolution comes out of it as trying to remember this issue everytime I do an editable install of a package has bitten me a few times.

qci-amos commented 1 year ago

It took me a fair amount of searching after a lot of mucking with setting interpreters to find this issue and the --config-settings editable_mode=strict solution... I've been a python developer for many years and I never "knew" there were different flavors of editable installs and wouldn't have guessed that pylance would care about which one I use! Nor would I have guessed that there would be persistent sticking issues on a 2.5 year old pep! It's very confusing! For years now, I've just accepted the "orange squiggles" on my python files as my fault.

I wonder if Pylance could add some text or a link to the reportMissingImports popup?

I just want to add that this solution is not currently sufficient for me. If I do "Go to Definition F12", it takes me to the /build/__editable__. folder, not to the directory in my vscode workspace. I assume this is very similar to the linked issue here, but in this case, I'm just using vscode (not a debugger). I don't want to mess with pathMappings, partly because I don't want to hand-edit that on principle, but also because I have many editable repos.

I found that --config-settings editable_mode=compat seems to solve both problems (pylance and f12).

I'm using

WSL
conda 23.7.4
py3.8.18
setuptools                  68.2.2
pip                         23.2.1

debonte commented 1 year ago

I wonder if Pylance could add some text or a link to the reportMissingImports popup?

@qci-amos, can you open a separate issue on this?

I just want to add that this solution is not currently sufficient for me...I found that --config-settings editable_mode=compat seems to solve both problems (pylance and f12).

There have been several similar issues filed and it doesn't surprise me that the user's chosen solution might differ in each. Our primary reference on PEP 660 import resolution is this doc page, which mentions both compat and strict.

ThiefMaster commented 3 months ago

I don't understand why PyLance cannot use a simple logic like this to handle PEP660-style editable install

import importlib
import re
import sys
from itertools import chain
from pathlib import Path

def get_editable_package_mappings():
    # XXX not using `ast` to parse the import from the pth file since it's way easier with a simple regex
    mappings = [
        importlib.import_module(m.group(1)).MAPPING
        for f in chain.from_iterable(
            Path(p).glob('__editable__.*.pth') for p in sys.path
        )
        if (m := re.search(r'import (__editable___\S+_finder)', f.read_text()))
    ]
    return mappings

This works perfectly fine to map package names to their source locations. Sure, it makes certain assumptions about the structure of those files, but since it's probably unlikely that those files will significantly chance any time soon (even more so if popular tools like PyLance use them), but it does the job.

In the typing mailing list discussion linked by @debonte a year ago, there's a mention of not being able to use anything but certain stdlib modules. My example above uses some more stdlib modules, but it would be fairly trivial to adapt the code and e.g. avoid importing the module but instead uses regex (and/or the ast module) to extract the mapping dict.

link89 commented 2 months ago

pip install -e . --config-settings editable_mode=strict not works for me.

debonte commented 2 months ago

pip install -e . --config-settings editable_mode=strict not works for me.

Does compat mode work for you? See https://microsoft.github.io/pyright/#/import-resolution?id=editable-installs for links to the pip/setuptools documentation.

If neither compat mode nor strict mode works, please file a new issue and we can investigate. Please include links to a repo that reproduces the issue you are seeing.

link89 commented 2 months ago

@debonte compat works!

stellaraccident commented 5 days ago

This still appears to be broken as of September 2024.

ThiefMaster commented 5 days ago

TBH I think the easiest solution, if it's your own package, is to just drop setuptools in favor of hatchling. AFAICT it only has advantages :)

stellaraccident commented 5 days ago

TBH I think the easiest solution, if it's your own package, is to just drop setuptools in favor of hatchling. AFAICT it only has advantages :)

Lots of things on the internet presume that tools work as advertised. I don't just need projects I control to work.

stellaraccident commented 5 days ago

The most reliable thing is just to stop expecting that any of it works and use a .env file.

debonte commented 5 days ago

We currently have no plans to make changes here. See my comment above for an explanation of how we got where we are today. If the packaging and typing communities can come to an agreement on a way that static analyzers (emphasis on static) can work better here, we would likely adopt that approach.

Also see Eric Traut's recent comment here: https://github.com/microsoft/pylance-release/discussions/6384#discussioncomment-10628431

stellaraccident commented 4 days ago

TBH I think the easiest solution, if it's your own package, is to just drop setuptools in favor of hatchling. AFAICT it only has advantages :)

For anyone else who would like to skip this research. From hatchling's readme: "If building extension modules is required then it is recommended that you continue using setuptools, or even other backends that specialize in interfacing with compilers."

stellaraccident commented 4 days ago

I ended up just writing a script that I can run in any repo to introspect the python environment and produce a .env file that gets the job done.

Such a thing could be better if the IDE had some config options that let me set a list of package_prefix->path mappings for a project vs the single flat list of paths.

This all seems like options that the static analyzer should support and then that the ide plug-in should be pragmatic about attempting to auto detect during environment setup.

Sure, I would have done the packaging specification differently, but I have little sympathy for the argument here that has produced a user hostile outcome for years from an IDE.

stellaraccident commented 4 days ago

I don't understand why PyLance cannot use a simple logic like this to handle PEP660-style editable install
import importlib
import re
import sys
from itertools import chain
from pathlib import Path

def get_editable_package_mappings():
    # XXX not using `ast` to parse the import from the pth file since it's way easier with a simple regex
    mappings = [
        importlib.import_module(m.group(1)).MAPPING
        for f in chain.from_iterable(
            Path(p).glob('__editable__.*.pth') for p in sys.path
        )
        if (m := re.search(r'import (__editable___\S+_finder)', f.read_text()))
    ]
    return mappings
This works perfectly fine to map package names to their source locations. Sure, it makes certain assumptions about the structure of those files, but since it's probably unlikely that those files will significantly chance any time soon (even more so if popular tools like PyLance use them), but it does the job.

In the typing mailing list discussion linked by @debonte a year ago, there's a mention of not being able to use anything but certain stdlib modules. My example above uses some more stdlib modules, but it would be fairly trivial to adapt the code and e.g. avoid importing the module but instead uses regex (and/or the ast module) to extract the mapping dict.

I realize that this style of solution can offend certain sensibilities, but if it were turned into a real workaround and recommended in a faq, it would have saved many people countless hours.

If there were a vscode setting that just took a json mapping of package prefix/path mappings, a small script like this could generate that. Then copy/paste, done.

Then maybe a couple of years from now the conversation resolves some declarative way to... and everyone is happy. I know this packaging stuff is a mess. Better workarounds and do-it-myself config options are the soup-du-joure.

FTR, there are significant usability hurdles to the strict mode recommendation -- so much so that I have spent a lot of time living with the orange squiggles as the better option. I've spent many hours trying to use it, and it is just better to slog on without ide integration.

microsoft / pylance-release