Conflicting dependencies culprit is not displayed

[NiklasRosenstein]

Description

I have a project with two packages. I upgraded their dependencies on databind.core and databind.json from >=0.8.0,<=1.0.0 to >=1.1.0,<2.0.0. I.e. in the setup.py of both packages, the install reqs are listed as

  'databind.json >=1.1.0,<2.0.0',
  'databind.core >=1.1.0,<2.0.0',

Now installing the packages again in develop mode gives me:

$ pip install -v -e spam-bot/ -e spam-bot-predicate-comments/
[ ... ]

The conflict is caused by:
    spam-bot 0.4.2 depends on databind.core<2.0.0 and >=1.1.0
    spam-bot-predicate-comments 0.4.2 depends on databind.core<2.0.0 and >=1.1.0
    databind-json 1.2.0 depends on databind.core<2.0.0 and >=1.2.0
    spam-bot 0.4.2 depends on databind.core<2.0.0 and >=1.1.0
    spam-bot-predicate-comments 0.4.2 depends on databind.core<2.0.0 and >=1.1.0
    databind-json 1.1.6 depends on databind.core<2.0.0 and >=1.1.6
    spam-bot 0.4.2 depends on databind.core<2.0.0 and >=1.1.0
    spam-bot-predicate-comments 0.4.2 depends on databind.core<2.0.0 and >=1.1.0
    databind-json 1.1.5 depends on databind.core<2.0.0 and >=1.1.5
    spam-bot 0.4.2 depends on databind.core<2.0.0 and >=1.1.0
    spam-bot-predicate-comments 0.4.2 depends on databind.core<2.0.0 and >=1.1.0
    databind-json 1.1.4 depends on databind.core<2.0.0 and >=1.1.4
    spam-bot 0.4.2 depends on databind.core<2.0.0 and >=1.1.0
    spam-bot-predicate-comments 0.4.2 depends on databind.core<2.0.0 and >=1.1.0
    databind-json 1.1.3 depends on databind.core<2.0.0 and >=1.1.3
    spam-bot 0.4.2 depends on databind.core<2.0.0 and >=1.1.0
    spam-bot-predicate-comments 0.4.2 depends on databind.core<2.0.0 and >=1.1.0
    databind-json 1.1.2 depends on databind.core<2.0.0 and >=1.1.2
    spam-bot 0.4.2 depends on databind.core<2.0.0 and >=1.1.0
    spam-bot-predicate-comments 0.4.2 depends on databind.core<2.0.0 and >=1.1.0
    databind-json 1.1.1 depends on databind.core<2.0.0 and >=1.1.1

To fix this you could try to:
1. loosen the range of package versions you've specified
2. remove package versions to allow pip attempt to solve the dependency conflict

ERROR: ResolutionImpossible: for help visit https://pip.pypa.io/en/latest/user_guide/#fixing-conflicting-dependencies

(Full logs: pip_install_logs_21_1_3.txt, pip_install_logs_21_2_4.txt)

But I don't see how this is a conflict.

• spam-bot 0.4.2 depends on databind.core<2.0.0 and >=1.1.0 *spam-bot-predicate-comments 0.4.2 depends on databind.core<2.0.0 and >=1.1.0

Ok they both depend on databind.core with the same version cosntraint.

• databind-json 1.2.0 depends on databind.core<2.0.0 and >=1.2.0

The version constraint of databind.json is slightly stricter, but would all three constraints are satisfiable with databind.core 1.2.0.

Now it turns out that thsi is an issue because there is also a requirement on databind.yaml >=0.1.2,<1.0.0 in the setup files which I did not notice earlier. That package is not compatible with databind.json or databind.core in those versions. Removing this package as a requirement resolves the conflict.

However the confusing this is that databind.yaml does not actually show up in the conflict error message at all.

Expected behavior

Pip is able to install the dependencies, in this case in version 1.2.0 for both databind.core and databind.json (latest on PyPI).

pip version

21.1.3, 21.2.4

Python version

3.9.6

OS

OSX

How to Reproduce

1. Install a package with the below setup file:

   import setuptools
   setuptools.setup(
     name='pkgb',
     version='0.0.0',
     install_requires=[
       'databind.json >=0.9.0,<1.0.0',
       'databind.core >=0.8.0,<1.0.0',
       'databind.yaml >=0.1.2,<1.0.0'
     ]
   )

   $ pip install .

2. Update the requirements for databind.core and databind.json and install again.

   import setuptools
   setuptools.setup(
     name='pkgb',
     version='0.0.0',
     install_requires=[
       'databind.json >=1.1.0,<2.0.0',
       'databind.core >=1.1.0,<2.0.0',
       'databind.yaml >=0.1.2,<1.0.0'
     ]
   )

   $ pip install .

3. Notice the conflict error but that it doesn't show the conflict with databind.yaml at all.

   ERROR: Cannot install pkgb and pkgb==0.0.0 because these package versions have conflicting dependencies.
   
   The conflict is caused by:
       pkgb 0.0.0 depends on databind.core<2.0.0 and >=1.1.0
       databind-json 1.2.0 depends on databind.core<2.0.0 and >=1.2.0
       pkgb 0.0.0 depends on databind.core<2.0.0 and >=1.1.0
       databind-json 1.1.6 depends on databind.core<2.0.0 and >=1.1.6
       pkgb 0.0.0 depends on databind.core<2.0.0 and >=1.1.0
       databind-json 1.1.5 depends on databind.core<2.0.0 and >=1.1.5
       pkgb 0.0.0 depends on databind.core<2.0.0 and >=1.1.0
       databind-json 1.1.4 depends on databind.core<2.0.0 and >=1.1.4
       pkgb 0.0.0 depends on databind.core<2.0.0 and >=1.1.0
       databind-json 1.1.3 depends on databind.core<2.0.0 and >=1.1.3
       pkgb 0.0.0 depends on databind.core<2.0.0 and >=1.1.0
       databind-json 1.1.2 depends on databind.core<2.0.0 and >=1.1.2
       pkgb 0.0.0 depends on databind.core<2.0.0 and >=1.1.0
       databind-json 1.1.1 depends on databind.core<2.0.0 and >=1.1.1
   
   To fix this you could try to:
   1. loosen the range of package versions you've specified
   2. remove package versions to allow pip attempt to solve the dependency conflict

Output

No response

Code of Conduct

• [X] I agree to follow the PSF Code of Conduct.

[SVAIRobotics]

I second @NiklasRosenstein and would like to report the same issue. It would be great to have this issue escalated further in terms of urgency since it's been ~4 months since he opened this.

pip version 21.3.1

Python version 3.8.5

OS Amazon Linux 2 (on AWS EC2 / Elastic Beanstalk)

Command run and Conflict reported by pip

I ran python3 -m pip install https://github.com/django-ai/djaiot/archive/dev.zip https://github.com/fastaiapi/fastaiapi/archive/dev.zip

The conflict reported by pip varies from run to run, but is of one same head-scratching pattern as follow:

The conflict is caused by: fastaiapi 0.0.0.dev0 depends on Django-MySQL<5.0.0 and >=4.3.0 djai 0.0.0.dev0 depends on Django-MySQL<5.0.0 and >=4.4.0

The conflict is caused by: fastaiapi 0.0.0.dev0 depends on NumPy<2.0.0 and >=1.20.3 djai 0.0.0.dev0 depends on NumPy<2.0.0 and >=1.21.5

As @NiklasRosenstein mentioned above, such different, but not conflicting, dependencies should not be problematic.

The real problem (which I deliberately constructed) which is not reported by pip is actually the following:

(ideal, correct conflict report) The conflict is caused by: fastaiapi 0.0.0.dev0 depends on HuggingFace-Hub<0.3.0 and >=0.2.1 djai 0.0.0.dev0 depends on HuggingFace-Hub<0.5.0 and >=0.4.0

[notatallshaw]

@pradyunsg @uranusjr You know if anyone has investigated this?

I've took a little look, this seems to be a reporting issue, but I've not dug that deep in to where the issue is happening.

[pombredanne]

As mentioned by @notatallshaw It could be similar to another issue I reported in #10824 I tried to reproduce this here.... and OMG this is fetching gigagbytes of wheels! ;) ... using pinned commit:

pip install https://github.com/Django-AI/DjAIoT/archive/6ee92ec22040d3d22619fdc6f7b1bf88121ab1bf.zip https://github.com/FastAIAPI/FastAIAPI/archive/0643928266ffeb5945dfb265f1c1ff5f95bada29.zip

This is failing for me with another reason though:

Collecting MySQLClient<3.0.0,>=2.1.0
  Downloading mysqlclient-2.1.0.tar.gz (87 kB)
     |████████████████████████████████| 87 kB 3.2 MB/s            
  Preparing metadata (setup.py) ... error
  ERROR: Command errored out with exit status 1:
   command: /home/pombreda/tmp/py38/venv/bin/python -c 'import io, os, sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-ufrxenk9/mysqlclient_924061556ed14eb196b8aacf42971ee8/setup.py'"'"'; __file__='"'"'/tmp/pip-install-ufrxenk9/mysqlclient_924061556ed14eb196b8aacf42971ee8/setup.py'"'"';f = getattr(tokenize, '"'"'open'"'"', open)(__file__) if os.path.exists(__file__) else io.StringIO('"'"'from setuptools import setup; setup()'"'"');code = f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' egg_info --egg-base /tmp/pip-pip-egg-info-rifv8ld8
       cwd: /tmp/pip-install-ufrxenk9/mysqlclient_924061556ed14eb196b8aacf42971ee8/
  Complete output (15 lines):
  /bin/sh: 1: mysql_config: not found
  /bin/sh: 1: mariadb_config: not found
  /bin/sh: 1: mysql_config: not found
  Traceback (most recent call last):
    File "<string>", line 1, in <module>
    File "/tmp/pip-install-ufrxenk9/mysqlclient_924061556ed14eb196b8aacf42971ee8/setup.py", line 15, in <module>
      metadata, options = get_config()
    File "/tmp/pip-install-ufrxenk9/mysqlclient_924061556ed14eb196b8aacf42971ee8/setup_posix.py", line 70, in get_config
      libs = mysql_config("libs")
    File "/tmp/pip-install-ufrxenk9/mysqlclient_924061556ed14eb196b8aacf42971ee8/setup_posix.py", line 31, in mysql_config
      raise OSError("{} not found".format(_mysql_config_path))
  OSError: mysql_config not found
  mysql_config --version
  mariadb_config --version
  mysql_config --libs
  ----------------------------------------
WARNING: Discarding https://files.pythonhosted.org/packages/de/79/d02be3cb942afda6c99ca207858847572e38146eb73a7c4bfe3bdf154626/mysqlclient-2.1.0.tar.gz#sha256=973235686f1b720536d417bf0a0d39b4ab3d5086b2b6ad5e6752393428c02b12 (from https://pypi.org/simple/mysqlclient/) (requires-python:>=3.5). Command errored out with exit status 1: python setup.py egg_info Check the logs for full command output.
INFO: pip is looking at multiple versions of <Python from Requires-Python> to determine which version is compatible with other requirements. This could take a while.
INFO: pip is looking at multiple versions of djaiot to determine which version is compatible with other requirements. This could take a while.
ERROR: Could not find a version that satisfies the requirement MySQLClient<3.0.0,>=2.1.0 (from fastaiapi) (from versions: 1.3.0, 1.3.1, 1.3.2, 1.3.3, 1.3.4, 1.3.5, 1.3.6, 1.3.7, 1.3.8, 1.3.9, 1.3.10, 1.3.11rc1, 1.3.11, 1.3.12, 1.3.13, 1.3.14, 1.4.0rc1, 1.4.0rc2, 1.4.0rc3, 1.4.0, 1.4.1, 1.4.2, 1.4.2.post1, 1.4.3, 1.4.4, 1.4.5, 1.4.6, 2.0.0, 2.0.1, 2.0.2, 2.0.3, 2.1.0rc1, 2.1.0)
ERROR: No matching distribution found for MySQLClient<3.0.0,>=2.1.0

... e.g. I do not have MySQL installed. Let me retry with mariadb installed :P

[pombredanne]

After an install of mariadb, this fails as expected with another message:

Collecting Auto-SKLearn<0.15.0,>=0.14.2
  Downloading auto-sklearn-0.14.2.tar.gz (6.3 MB)
     |████████████████████████████████| 6.3 MB 2.3 MB/s            
  Preparing metadata (setup.py) ... done
Collecting dask<2021.07
  Downloading dask-2021.6.2-py3-none-any.whl (973 kB)
     |████████████████████████████████| 973 kB 4.4 MB/s            
Collecting distributed<2021.07,>=2.2.0
  Downloading distributed-2021.6.2-py3-none-any.whl (722 kB)
     |████████████████████████████████| 722 kB 4.4 MB/s            
INFO: pip is looking at multiple versions of asgiref to determine which version is compatible with other requirements. This could take a while.
Collecting ASGIRef<4.0.0,>=3.4.1
  Using cached asgiref-3.4.1-py3-none-any.whl (25 kB)
INFO: pip is looking at multiple versions of fastaiapi to determine which version is compatible with other requirements. This could take a while.
INFO: pip is looking at multiple versions of <Python from Requires-Python> to determine which version is compatible with other requirements. This could take a while.
INFO: pip is looking at multiple versions of djaiot to determine which version is compatible with other requirements. This could take a while.
ERROR: Cannot install djaiot and fastaiapi==0.0.0.dev0 because these package versions have conflicting dependencies.

The conflict is caused by:
    fastaiapi 0.0.0.dev0 depends on Auto-SKLearn<0.15.0 and >=0.14.2
    djai 0.0.0.dev0 depends on Auto-SKLearn<0.15.0 and >=0.14.3

To fix this you could try to:
1. loosen the range of package versions you've specified
2. remove package versions to allow pip attempt to solve the dependency conflict

ERROR: ResolutionImpossible: for help visit https://pip.pypa.io/en/latest/user_guide/#fixing-conflicting-dependencies

Yet, $ pip install Auto-SKLearn==0.14.3 works perfectly well there.

So my conclusion is that the pip resolver does not work when there are multiple perfectly overlapping version ranges.
How can I disable entirely the resolver to go back to something saner?

[notatallshaw]

So my conclusion is that the pip resolver does not work when there are multiple perfectly overlapping version ranges. How can I disable entirely the resolver to go back to something saner?

I'm not convinced that it's a resolver issue and instead a reporting issue, i.e. the pip resolver has found something but the reporting has a bug in it. But I've not spent any time yet trying to debug,

The example in https://github.com/pypa/pip/issues/10391#issuecomment-1010628198 has a real conflict, it's just not reported correctly.

[pombredanne]

@notatallshaw ok :)

FWIW running:

pip install Auto-SKLearn==0.14.3

Then again:

pip install https://github.com/Django-AI/DjAIoT/archive/6ee92ec22040d3d22619fdc6f7b1bf88121ab1bf.zip https://github.com/FastAIAPI/FastAIAPI/archive/0643928266ffeb5945dfb265f1c1ff5f95bada29.zip

yield a new error, as hard to diagnose as before.

ERROR: Cannot install djaiot and fastaiapi==0.0.0.dev0 because these package versions have conflicting dependencies.

The conflict is caused by:
    fastaiapi 0.0.0.dev0 depends on Ray<2.0.0 and >=1.9.1; python_version < "3.10"
    djai 0.0.0.dev0 depends on Ray<2.0.0 and >=1.9.2; python_version < "3.10"

To fix this you could try to:
1. loosen the range of package versions you've specified
2. remove package versions to allow pip attempt to solve the dependency conflict

ERROR: ResolutionImpossible: for help visit https://pip.pypa.io/en/latest/user_guide/#fixing-conflicting-dependencies

[pombredanne]

@notatallshaw you wrote:

I'm not convinced that it's a resolver issue and instead a reporting issue, i.e. the pip resolver has found something but the reporting has a bug in it.

I am not convinced either... but how could this be debugged? Is there some tracing or logging mode where the resolver could spit more details?

[pombredanne]

(Side note/rant: The two top level packages listed here are both built with poetry that (IMHO mistakenly) promotes by default to have upper versions capped and this is likely to be a source of infinite head scratching without a proper reporting of resolutions.)

( https://github.com/Django-AI/DjAIoT/archive/6ee92ec22040d3d22619fdc6f7b1bf88121ab1bf.zip and https://github.com/FastAIAPI/FastAIAPI/archive/0643928266ffeb5945dfb265f1c1ff5f95bada29.zip )

[notatallshaw]

I am not convinced either... but how could this be debugged? Is there some tracing or logging mode where the resolver could spit more details?

I checked if the pip verbose flags helped here (-v, -vv, & -vvv) but I didn't notice anything particularly helpful testing against the test case in https://github.com/pypa/pip/issues/10391#issuecomment-1010628198. I think there are some special environmental variables to get more information, but I don't remember what they are, maybe a pip maintainer can chime in there is anything helpful.

If I get some time this evening I will manually add some breakpoints (or better yet learn to use the --debug mode), I do notice testing with verbose mode enabled that it appears that the ResolutionImpossible error that resolvelib raises appears to contain different information which surprises me and may indicate that resolvelib is providing the wrong information.

Here are some outputs of running the same command:

2022-01-25T12:37:00,061     raise ResolutionImpossible(self.state.backtrack_causes)
2022-01-25T12:37:00,061 pip._vendor.resolvelib.resolvers.ResolutionImpossible: [RequirementInformation(requirement=SpecifierRequirement('MariaDB-DynCol<4.0.0,>=3.4.0'), parent=LinkCandidate('https://github.com/fastaiapi/fastaiapi/archive/dev.zip')), RequirementInformation(requirement=SpecifierRequirement('MariaDB-DynCol<4.0.0,>=3.5.0'), parent=LinkCandidate('https://GitHub.com/Django-AI/DjAI/archive/dev.zip'))]

2022-01-25T12:39:10,065     raise ResolutionImpossible(self.state.backtrack_causes)
2022-01-25T12:39:10,065 pip._vendor.resolvelib.resolvers.ResolutionImpossible: [RequirementInformation(requirement=SpecifierRequirement('Ray<2.0.0,>=1.9.1; python_version < "3.10"'), parent=LinkCandidate('https://github.com/fastaiapi/fastaiapi/archive/dev.zip')), RequirementInformation(requirement=SpecifierRequirement('Ray<2.0.0,>=1.9.2; python_version < "3.10"'), parent=LinkCandidate('https://GitHub.com/Django-AI/DjAI/archive/dev.zip'))]

2022-01-25T12:41:55,674     raise ResolutionImpossible(self.state.backtrack_causes)
2022-01-25T12:41:55,674 pip._vendor.resolvelib.resolvers.ResolutionImpossible: [RequirementInformation(requirement=SpecifierRequirement('HuggingFace-Hub<0.3.0,>=0.2.1'), parent=LinkCandidate('https://github.com/fastaiapi/fastaiapi/archive/dev.zip')), RequirementInformation(requirement=SpecifierRequirement('huggingface-hub<1.0.0,>=0.1.0'), parent=LinkCandidate('https://files.pythonhosted.org/packages/28/5c/afd4d9f8b276579334e5f4db7f6fdf2fa43ac9f957d88573a7cc4f5736f6/datasets-1.17.0-py3-none-any.whl#sha256=8899a69914c7c19029a085e994f9b27dfeacd24b0d04c792e41e65cf1b033cab (from https://pypi.org/simple/datasets/)')), RequirementInformation(requirement=SpecifierRequirement('huggingface-hub<1.0,>=0.1.0'), parent=LinkCandidate('https://files.pythonhosted.org/packages/4a/7f/f1c28621af0d74794b18cbe5534ec7565ee782ba48257d08ec264bc4aacb/transformers-4.15.0-py3-none-any.whl#sha256=148fdfc68d703d61f1651c98b228f10c36620eef41722911ee531bfdd4941f18 (from https://pypi.org/simple/transformers/) (requires-python:>=3.6.0)')), RequirementInformation(requirement=SpecifierRequirement('HuggingFace-Hub<0.5.0,>=0.4.0'), parent=LinkCandidate('https://GitHub.com/Django-AI/DjAI/archive/dev.zip'))]

Interestingly the last one raised the correct conflict (but with no difference in how I ran the commands so it just appears random if you get it or not):

The conflict is caused by:
    fastaiapi 0.0.0.dev0 depends on HuggingFace-Hub<0.3.0 and >=0.2.1
    datasets 1.17.0 depends on huggingface-hub<1.0.0 and >=0.1.0
    transformers 4.15.0 depends on huggingface-hub<1.0 and >=0.1.0
    djai 0.0.0.dev0 depends on HuggingFace-Hub<0.5.0 and >=0.4.0

Unfortunately @pombredanne I can not test your examples in https://github.com/pypa/pip/issues/10824 as I do not have any Mac hardware, so at the moment I am only assuming these two issues are related.

[pombredanne]

Unfortunately @pombredanne I can not test your examples in #10824 as I do not have any Mac hardware, so at the moment I am only assuming these two issues are related.

Same here... I am a linux person :)

[notatallshaw]

I spent a bit of time debugging and my only finding is it's not just the conflict message at the end but the requirements are resolved in a different order. which I don't think should happen.

I am unsure if this is a bug in PipProvider.get_preference on pip's side or some logic issue on resolvelib side or something else.

[pombredanne]

@notatallshaw Thanks! I surmise that some unordered collection usage is the culprit ... like a set... it would be the perfect candidate for randomization ;)

I can see resolvelib using these in these places:

• https://github.com/sarugaku/resolvelib/blob/master/src/resolvelib/resolvers.py#L425
• https://github.com/sarugaku/resolvelib/blob/32ebd52762deffceb7f6ad1d9a71cc8cc7794629/src/resolvelib/structs.py#L10

and in pip:

• https://github.com/pypa/pip/blob/ba979fd4b1094a4145731363b37a91b59fe4d4d5/src/pip/_internal/resolution/resolvelib/resolver.py#L199

The legacy resolver was sorting sets like in https://github.com/pypa/pip/blob/e0c7f24ee4fd9d3279b25acfd978c080a212b5c1/src/pip/_internal/resolution/legacy/resolver.py#L421 whereas I do not as much sorting taking place with the new resolver.

Eventually this could mean that the ordering of the directed graph is pseudo-random and the resolution therefore would be too.

[pombredanne]

So in all cases there are likely two errors:

• error messages that do not provide proper actionable information making the issue impossible to diagnose and fix easily
• randomized resolution and installation order that compounds the lack of proper error message above
• (plus some possible extra underlying resolvelib and pip issue TBD)

[pradyunsg]

Can someone please run with PIP_RESOLVER_DEBUG=1 in the environment, and post the output of a couple of runs as GitHub Gists?

I've not had time to reproduce this on a compatible machine, and those logs would likely be insightful.

[notatallshaw]

Can someone please run with PIP_RESOLVER_DEBUG=1 in the environment, and post the output of a couple of runs as GitHub Gists?

Here are 3 runs: https://gist.github.com/notatallshaw/807eae4ac1735da1f19a90e17922d346

All 3 are truncated to the first 5000 lines, as no matter how many runs I did none of the logs ended up being small enough to upload to gist (varied from ~18 MBs to > 1 GB). However the first difference between the resolution step was always in the first 100 lines, usually right after Reporter.starting_round(2) but sometimes as late as right after Reporter.starting_round(4).

[notatallshaw]

With Pip 23.1 there have been recent significant changes to resolvelib, including fixing a known logical errors, this is now the conflict error message I get:

The conflict is caused by:
    pkgb 0.0.0 depends on databind.core<2.0.0 and >=1.1.0
    databind-json 1.1.0 depends on databind.core<2.0.0 and >=1.1.0
    databind-yaml 0.1.3 depends on databind.core<1.0.0 and >=0.1.0
    pkgb 0.0.0 depends on databind.core<2.0.0 and >=1.1.0
    databind-json 1.1.0 depends on databind.core<2.0.0 and >=1.1.0
    databind-yaml 0.1.2 depends on databind.core<1.0.0 and >=0.1.0

This seems correct right?

[pradyunsg]

It does!