`mypy` breaks when installed with `pip install --prefix`

[P403n1x87]

Bug Report

Let me introduce some shorthand notation for the sake of clarity. Let V be a virtual environment, Vsp the site-packages location within V, P a prefix path and Psp the site-packages location within P.

Now suppose that mypy is installed in P with pip install --prefix P mypy from within V. When invoking mypy from V, with mypy --install-types, the missing types are installed in Vsp, and this leads to errors of the kind

tests/test_unit.py:12: error: Untyped decorator makes function "current_interpreter" untyped
tests/test_unit.py:17: error: Untyped decorator makes function "current_venv" untyped
tests/test_integration.py:39: error: Untyped decorator makes function "tmp_run" untyped
tests/test_cli.py:18: error: Untyped decorator makes function "cli" untyped
tests/test_cli.py:273: error: Untyped decorator makes function "test_run_suites_cmdargs" untyped
Found 5 errors in 3 files (checked 10 source files)

(note that when mypy is installed in V rather than in P, no errors are reported, as expected). Setting MYPYPATH to include Vsp gives

.riot/venv_py394/lib/python3.9/site-packages is in the MYPYPATH. Please remove it.
See https://mypy.readthedocs.io/en/stable/running_mypy.html#how-mypy-handles-imports for more info

which is documented behaviour. Installing the missing types by hand in P from V with pip install --prefix type-... doesn't help:

riot/cli.py:9: error: Library stubs not installed for "pkg_resources" (or incompatible with Python 3.9)
riot/cli.py:9: note: Hint: "python3 -m pip install types-setuptools"
riot/cli.py:9: note: (or run "mypy --install-types" to install all missing stub packages)
riot/cli.py:9: note: See https://mypy.readthedocs.io/en/stable/running_mypy.html#missing-imports

Setting MYPYPATH to include Psp gives

mypy: ".riot/venv_py394_types-mock_types-setuptools_mypy_pytest/lib/python3.9/site-packages/typing_extensions.py" shadows library module "typing_extensions"
note: A user-defined top-level module with name "typing_extensions" is not supported

So there doesn't seem to be a way to install mypy in a prefix

To Reproduce

See the above description of the issue

Expected Behavior

I would expect mypy to work even if I'd have to set MYPYPATH to something

Actual Behavior

No possible way of using mypy from a custom prefix.

Your Environment

• Mypy version used: 0.910
• Mypy command-line flags: --install-types --non-interactive
• Mypy configuration options from mypy.ini (and other config files):

  [mypy]
  ignore_missing_imports = true
  disallow_incomplete_defs = true
  disallow_untyped_decorators = true
  warn_unused_ignores = true
  warn_unused_configs = true
  disallow_any_generics = true
  warn_redundant_casts = true
  warn_return_any = true
  strict_equality = true
  no_implicit_optional = true
  files = riot,tests

• Python version used: 3.9
• Operating system and version: macOS Big Sur

[ethanhs]

Duplicate of https://github.com/python/mypy/issues/4380 but that user seems to have been fine with using a virtualenv. It seems that you are using prefix inside a venv already. Is there a reason you can't put the virtualenv in the prefix you want?

[P403n1x87]

The reason is that we have a nested data structure that describes Python packages that need to be installed at each nesting level. Deeper levels should override the previous ones, so the packages are installed in a specific prefix at each level. The root level represents the base virtual env and nothing gets installed in the default prefix, so this is why mypy is installed in a non-default prefix even at the first level after root. If we wanted to install mypy in the default prefix we would have to hard-code a specific rule for it, which would be a bit awkward.

[ethanhs]

Hm, I'm not sure I entirely understand your use case, but why can't you just install mypy and other tools in separate venvs and execute them like venv/bin/mypy?

Otherwise, if you would like to propose a change that would fix mypy with --prefix, I will review it.

[P403n1x87]

why can't you just install mypy and other tools in separate venvs and execute them like venv/bin/mypy?

We can of course and that's more or less what we currently do, but we were hoping to reduce the number of venvs we generate as we only need one if we use prefixes and judiciously set the PYTHONPATH and PATH. This optimisation would save us memory and time in CI.

If it helps, this is the kind of structure that we have

Venv(
    pys=["3.8"],
    name="foo",
    pkgs={
        "pytest": latest  # this is just an example
    },
    venvs=[
        Venv(
            name="bar",
            command="mypy --install-types --non-interactive .",
            pkgs={
                "mypy": ["==0.900", latest]
            },
        ),
    ],
)

This generates two venv instances where we have a common "layer" with just pytest and two layers with mypy==0.900 and mypy respectively on top. These dependencies are installed in their own prefixes inside a 3.8 venv. Currently, we generate all the intermediate venvs that require packages and activate the leaf ones, but by setting the PYTHONPATH and PATH we could use just the base one, thus saving memory and time.

[P403n1x87]

@ethanhs I have opened #10833 and would love some feedback to see if that's going in the right direction. If it's looking good, I would also appreciate some guidelines for adding a test case. This would require installing mypy outside the default prefix.