search

devcontainers / images

Repository for pre-built dev container images published under mcr.microsoft.com/devcontainers
https://containers.dev
MIT License
1.25k stars 459 forks source link

Python standard library Modulefinder not working with editable installed packages #1077

Open micdet-delen opened 4 months ago

micdet-delen commented 4 months ago

Hello,

This is related to the Python images: https://github.com/devcontainers/images/tree/main/src/python

When having a Python package installed as editable (pip install -e ...) the standard library Modulefinder cannot find the package.

See also: https://github.com/thebjorn/pydeps/issues/222

I'll try to recap/provide a how-to-reproduce:

1) install a (custom) package as editable: pip install -e mypackage 2) have it imported by a simple test file: echo "import mypackage" > test.py 3) verify that the package can indeed be imported by python: python test.py (should not throw import errors) 4) have modulefinder (not) find the package:

$ python -m modulefinder test.py 

  Name                      File
  ----                      ----
m __main__                  test.py

Missing modules:
? mypackage imported from __main__

This does not occur when using the Debian provided Python version.

In the latest version that I've tested (3.12-bookworm) this also does not occur when installing the package normally.

In a previous version however (3.11-bullseye) installing it normally threw a stacktrace, ending in:

massive stacktrace before this
  File "/usr/local/lib/python3.11/modulefinder.py", line 308, in import_module
    fp, pathname, stuff = self.find_module(partname,
                          ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/modulefinder.py", line 489, in find_module
    return _find_module(name, path)
           ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/modulefinder.py", line 69, in _find_module
    if spec.loader.is_package(name):
       ^^^^^^^^^^^^^^^^^^^^^^
AttributeError: 'NoneType' object has no attribute 'is_package'
samruddhikhandale commented 4 months ago

This issue seems to be related to how Python's modulefinder handles editable packages installed via pip install -e.

In Python, an editable install (pip install -e) creates a link in your site-packages directory to your project's source code directory. This allows you to make changes to your package without having to reinstall it. However, modulefinder might not be able to find these packages because it's looking for actual package files, not links.

The error message AttributeError: 'NoneType' object has no attribute 'is_package' suggests that modulefinder is trying to call the is_package method on a None object, which likely means it couldn't find the package you're trying to import.

As a workaround, you could try to install the package normally (without -e) before running modulefinder. If you need to make changes to the package, you can reinstall it after making your changes.

If this doesn't work, you might want to consider using a different tool for finding modules, or filing a bug report with the Python team.