mypy doesn't find stubs when on PYTHONPATH

[jaraco]

Probably an upstream issue, but first encountered here, so reporting the details. Running a check on a file importing zope.interface will result in an error "Skipping analyzing zope.interface".

draft $ cat mypy.ini
[mypy]
plugins=mypy_zope:plugin
draft $ cat > interface.py
from zope.interface.interface import Interface

class X(Interface):
    pass
draft $ python -m venv venv
draft $ venv/bin/pip install -q mypy-zope
draft $ venv/bin/pip uninstall -q -y mypy-zope
draft $ venv/bin/pip install -q -t libs --no-deps mypy-zope
draft $ env PYTHONPATH=libs venv/bin/python -m mypy interface.py
interface.py:1: error: Skipping analyzing 'zope.interface.interface': found module but no type hints or library stubs
interface.py:1: note: See https://mypy.readthedocs.io/en/latest/running_mypy.html#missing-imports
Found 1 error in 1 file (checked 1 source file)

Even though the stubs are there:

draft $ tree libs/zope-stubs/
libs/zope-stubs/
├── __init__.pyi
├── interface
│   ├── __init__.pyi
│   ├── _compat.pyi
│   ├── _flatten.pyi
│   ├── _zope_interface_coptimizations.pyi
│   ├── adapter.pyi
│   ├── advice.pyi
│   ├── common
│   │   ├── __init__.pyi
│   │   ├── idatetime.pyi
│   │   ├── interfaces.pyi
│   │   ├── mapping.pyi
│   │   └── sequence.pyi
│   ├── declarations.pyi
│   ├── document.pyi
│   ├── exceptions.pyi
│   ├── interface.pyi
│   ├── interfaces.pyi
│   ├── registry.pyi
│   ├── ro.pyi
│   └── verify.pyi
└── schema
    ├── __init__.pyi
    ├── _bootstrapfields.pyi
    ├── _bootstrapinterfaces.pyi
    ├── _compat.pyi
    ├── _field.pyi
    ├── _messageid.pyi
    ├── _schema.pyi
    ├── accessors.pyi
    ├── fieldproperty.pyi
    ├── interfaces.pyi
    └── vocabulary.pyi

3 directories, 31 files

Installing the plugin directly into the virtualenv seems to work fine.

draft $ venv/bin/pip install -q mypy-zope
draft $ venv/bin/python -m mypy interface.py
Success: no issues found in 1 source file

[jaraco]

Looks like this issue goes back to 2013 (python/mypy#175).

[kedder]

Yeah, the plugin doesn't do anything special about module lookups (and probably shouldn't do).

Instead of setting PYTHONPATH, you can try setting mypy_path=libs in your mypy.ini. See https://mypy.readthedocs.io/en/stable/config_file.html#confval-mypy_path

[jaraco]

Adding mypy_path doesn't seem to help:

draft $ cat mypy.ini
[mypy]
plugins=mypy_zope:plugin
mypy_path=libs
draft $ env PYTHONPATH=libs venv/bin/python -m mypy interface.py
interface.py:1: error: Skipping analyzing 'zope.interface.interface': found module but no type hints or library stubs
interface.py:1: note: See https://mypy.readthedocs.io/en/latest/running_mypy.html#missing-imports
Found 1 error in 1 file (checked 1 source file)

Omitting PYTHONPATH altogether as suggested only results in the plugin failing to load:

draft $ venv/bin/python -m mypy interface.py
Found 1 error in 1 file (checked 1 source file)
mypy.ini:2: error: Error importing plugin 'mypy_zope': No module named 'mypy_zope'

[jaraco]

I even tried using an absolute path to libs in the mypy_path directive, but that produced the same results.

[jaraco]

I also tried setting MYPY_PATH=libs in the environment, but that was no help either.

[kedder]

Hm, I'm not sure what's your problem now. Stubs for zope.interface and zope.schema are included in mypy-zope, so you should not provide them yourself. Why are you trying to include them?

[jaraco]

I'm not trying to provide stubs, only trying to consume them from mypy-zope when mypy-zope is not installed in system-site-packages (only available on sys.path). I'm pretty sure it's an upstream issue with mypy itself failing to load stubs that aren't on some blessed path. I'm trying to include the stubs because the mypy checks fail without the stubs. Maybe I don't understand the question.

[kedder]

Is there a simple way for me to repro the issue?

[jaraco]

Would you like a Dockerfile that implements the repro in the OP?

[kedder]

That would work

[jaraco]

Here's the Dockerfile:

from jaraco/multipy-tox
RUN echo "[mypy]\nplugins=mypy_zope:plugin" > mypy.ini
RUN echo "from zope.interface.interface import Interface\nclass X(Interface):\n    pass" > interface.py
RUN pip install mypy-zope
RUN pip uninstall -y mypy-zope
RUN pip install -t libs --no-deps mypy-zope
CMD env PYTHONPATH=libs mypy interface.py

If you use docker build on that file, then docker run the build, you should see the reported error:

test $ docker run @($(docker build -q .).strip())  # xonsh syntax
interface.py:1: error: Skipping analyzing 'zope.interface.interface': found module but no type hints or library stubs
interface.py:1: note: See https://mypy.readthedocs.io/en/latest/running_mypy.html#missing-imports
Found 1 error in 1 file (checked 1 source file)

That's xonsh syntax, but assuming you're using a bash-like shell, you can probably do:

$ docker run $(docker build -q .)

The docker example doesn't use any virtualenvs but just installs everything (except mypy-zope) in the system site-packages. You should be able to docker run -it $(docker build -q .) bash to inspect and interact with the environment.

[kedder]

Oh, I see. I'll try to play with it tomorrow.

[kedder]

So, a bit later than I would've liked, but I did look into this. Apparently mypy only looks for stub dirs in site-packages dir, i.e. ones that are returned by site.getsitepackages() and site.getusersitepackages(). And these do not honor PYTHONPATH.

If you want to dig deeper, the third parameter to SearchPaths is used for stub lookup https://github.com/python/mypy/blob/master/mypy/modulefinder.py#L626-L629