pypa / pip

The Python package installer
https://pip.pypa.io/
MIT License
9.52k stars 3.03k forks source link

Add `--ignore` flag to `pip check` #11157

Open wolfv opened 2 years ago

wolfv commented 2 years ago

What's the problem this feature will solve?

We currently use pip check quite a bit in conda-forge to make sure that the dependency specifications that the dependencies that we have are correct.

However, some packages pip sometimes specifies as dependencies can also be proper "system"-packages which do not register as pip/python packages. That makes it hard to run pip check because of these "false negatives". Some examples are cmake, ninja, and likely some others.

Describe the solution you'd like

If we could add a --ignore flag and a corresponding environment variable that could help us quite a bit.

Alternative Solutions

Another solution would be to make pip check check if e.g. the ninja binary is on the $PATH -- but that's brittle. Another solution would be to register system packages as pip installed packages but that's also not a proper solution.

Additional context

https://github.com/FFY00/meson-python/issues/60

Code of Conduct

rgommers commented 2 years ago

Thanks @wolfv. +1 to this idea.

To save maintainers some following links, what came up in conda-forge was a recipe that had to explicitly apply this patch in order for pip check to not fail in CI:

diff --git a/PKG-INFO b/PKG-INFO
index 40355c0..e0f6a39 100644
--- a/PKG-INFO
+++ b/PKG-INFO
@@ -14,7 +14,6 @@ Project-URL: Changelog, https://mesonpy.readthedocs.io/en/latest/changelog.html
 Requires-Python: >=3.7
 Requires-Dist: colorama; os_name == "nt"
 Requires-Dist: meson>=0.60.0
-Requires-Dist: ninja
 Requires-Dist: pep621>=0.3.0
 Requires-Dist: tomli>=1.0.0
 Requires-Dist: typing-extensions>=3.7.4; python_version < "3.8"
diff --git a/pyproject.toml b/pyproject.toml
index d61a210..b66565a 100644
--- a/pyproject.toml
+++ b/pyproject.toml
@@ -3,7 +3,6 @@ build-backend = 'mesonpy'
 backend-path = ['.']
 requires = [
   'meson>=0.62.0',
-  'ninja',
   'pep621>=0.3.0',
   'tomli>=1.0.0',
   'typing-extensions>=3.7.4; python_version<"3.8"',
@@ -28,7 +27,6 @@ classifiers = [
 dependencies = [
   'colorama; os_name == "nt"',
   'meson>=0.60.0',
-  'ninja',
   'pep621>=0.3.0', # not a hard dependency, only needed for projects that use PEP 621 metadata
   'tomli>=1.0.0',
   'typing-extensions>=3.7.4; python_version<"3.8"',

It is not great that redistributors have to carry patches like that, and the ability to ignore checking for packages on the command line like @wolfv proposes would be a lot nicer.

Some examples are cmake, ninja, and likely some others.

patchelf is another one that comes to mind. And, once we break it out of numpy/scipy as a standalone native library rather than vendor it inside wheels (as we plan to do in the near future), openblas.

pradyunsg commented 2 years ago

This seems reasonable! A PR implementing this will be welcome. :)

q0w commented 2 years ago

--ignore flag should accept list of packages or files, containing list of packages?

wolfv commented 2 years ago

I don't know how pip does this usually but maybe a comma seperated list of packages? I think it could be good to also read an env var such as PIP_CHECK_IGNORE_PACKAGES=ninja,cmake,...

q0w commented 2 years ago

if you want env PIP_CHECK_IGNORE_PACKAGES, then flag should be --check-ignore-packages (like pip check --check-ignore-packages) because pip automatically converts all flags to env variables.

goanpeca commented 1 year ago

Bumping this 👍🏼 👁️

uranusjr commented 1 year ago

No need to bump, a pull requests is always welcomed.

goanpeca commented 1 year ago

There is one in place, looking forward to it getting merged soon https://github.com/pypa/pip/pull/11159 :)

dlqqq commented 2 days ago

This issue can also occur if your application requires a module that is distributed under different package names. The Jupyter AI Conda recipe is experiencing this issue.

We require the faiss module, which is only distributed through the faiss-cpu package on PyPI, so we have to list faiss-cpu as a required dependency in pyproject.toml. Installing faiss-cpu from Conda Forge also provides the faiss module, but installs it under a package named faiss, which is built directly from the source code on GitHub. This causes pip check to fail when Jupyter AI is installed from Conda, since faiss is installed but not faiss-cpu.

Is there any workaround for this while we await #11159? For example, is there a way to modify pyproject.toml to "require a module but don't require the PyPI package providing the module"?