Allow/disallow pep561 packages

[ethanhs]

@alexander255 has requested a flag --pep561-override to allow listing packages as pep 561 compatible when they are not. This got me thinking and I think the best UX for this would be a pair of flags: --allow-installed-package NAME and --disallow-installed-package NAME (or something like that).

The idea is to solve two problems:

Some people publish PEP 561 packages that are either too unstable or flat out broken for some users. This allows the user to disable the package until things are fixed.

At the other end of the spectrum, some packagers are conservative on adding types, and while they exist, they are not public (yet). Some users would still like to use them.

This would eliminate --no-site-packages, which is a rather blunt instrument.

[JukkaL]

I can see how this could be useful., and this shouldn't be hard to implement

Random thoughts:

• What about --use-installed-package X as the option name? The inverse would be --ignore-installed-package X, perhaps. More ideas:
  • --enable-installed-package X and --disable-installed-package X
  • --use-site-package X and --no-site-package X
  • --site-package X and --no-site-package X
  • --installed-package X and --no-installed-package X
• We can start with --use-site-package (or whatever it would be called) and the inverse can be implemented later. The inverse seems less commonly useful.
• Should this work for subpackages? What about things like --use-site-package google.protobuf?

[ethanhs]

I think I like either --enable-installed-packages X or --use-installed-packages X, ideally I think this should be a list of packages, as people may want to allowlist/denylist groups of packages.

[msullivan]

I am a "-0" on this; I don't like adding more complexity to our import system. But also I'd be happy to merge it. Let's make a decision on it so we can resolve #8512.

[ethanhs]

Yeah I'm kinda -0 on this too. The complexity would be unfortunate, but I can kinda see the use case.

[Vozf]

Just to clarify all selected flags will be usable via mypy.ini, correct?

[embray]

:+1: for this. I am using some packages that are fully typed and are PEP-561 compatible in principle (they have a py.typed) but due to a packaging bug (third-party package, outside my control) were missing the py.typed file when installed.

I can of course work around it by creating the py.typed file in the site-packages/<package_name> but it would be more convenient if I could tell mypy "yes, this package really does have type hints".

[davidism]

Adding my proposal from #10783, which concerns packages that have typing information but forgot to include a py.typed file. Many packages seem to make this mistake, and some are lightly maintained and haven't responded to bug reports or made new releases.

This should be a configuration option mirroring ignore_missing_imports, such as allow_missing_pytyped. Currently if a package is missing py.typed, my only recourse is to add

[mypy-missing.*]
ignore_missing_imports = True

Instead, since I know this package has typing information but forgot the py.typed file, I want to add

[mypy-missing.*]
allow_missing_pytyped = True

[erictraut]

I can see the appeal of providing a way to work around library shortcomings like this, but it would serve the entire Python community better if you would work with the library maintainer to fix the problem at the source. If the library is no longer maintained, perhaps a fork is the best answer?

[davidism]

The library is maintained, just slowly because the maintainer is also a maintainer of many other packages. I've already reached out to projects and even provided PRs sometimes, but I have no control over when they have time to review, merge, and release new versions. For the same reason, I have no desire to fork, as I already maintain enough projects as it is.

[hauntsaninja]

I believe the disallow use case can already be accomplished using something like:

[mypy]
[mypy-numpy.*]
follow_imports = skip
follow_imports_for_stubs = True

[davidism]

What is the difference between follow_imports = skip and ignore_missing_imports = true? Does that stubs option also apply for packages that have type annotations in the code, not stub files?

[davidism]

@hauntsaninja ping, could you respond to my questions about your proposed workaround?

[hauntsaninja]

ignore_missing_imports will only ignore missing imports. It will still resolve imports that exist, see example follow_imports = skip instead will make everything Any, see https://mypy.readthedocs.io/en/stable/running_mypy.html#following-imports

I believe you'd only need follow_imports_for_stubs = True for PEP 561 packages that contain pyi files.

[davidism]

follow_imports = skip does not sound like what I need. It will cause packages without a py.typed marker to be treated as Any, but I want them to be treated as if they have a py.typed marker.

[hauntsaninja]

You're right, sorry for confusion. The thing that triggered me to post ^^ was someone on gitter asking for the equivalent of --disallow-installed-package in the original post — I think some release of numpy had subpar types which they wished to ignore — for which I think my solution works. Clearly I should have disambiguated that since I confused myself a little just now too!

[Eyal-Shalev]

Are there any updates on this?

[and3rson]

Just found this while searching for a way to force mypy to treat a third-party library as py.typed. Any plans on this?

[jeremyephron]

This would be extremely useful feature to include. mypy typically does a good job checking many non PEP 561 compliant packages, and this can more readily enable workflows where you opt out 3rd-party packages from being checked by adding a follow_imports = skip for the specific packages that are not suitable to be checked.

[sigma67]

Similarly to @and3rson , I also came here because a third party library is missing a py.typed which I have no way of fixing in the short term

[DeinAlptraum]

I've opened a PR to implement this at #17712, would be great if someone could review this

[Priyansh121096]

Similarly to @and3rson , I also came here because a third party library is missing a py.typed which I have no way of fixing in the short term

+1 for this.