search

pradyunsg / zazo

Pure Python Dependency Resolution (see https://github.com/sarugaku/resolvelib/ instead)
MIT License
80 stars 7 forks source link

Pip 20.2.2 with use-feature=2020-resolver is not able to find a conflict in an hour #21

Open m-aciek opened 4 years ago

m-aciek commented 4 years ago

Pip loops itself after invoking pip install -r requirements.txt --use-feature=2020-resolver (more than 1 hour):

…
Requirement already satisfied: six>=1.4.1 in /usr/local/lib/python3.7/site-packages (from cryptography==3.0->-r requirements.txt (line 4)) (1.15.0)
Requirement already satisfied: internal-lib3<5dev,>=4.0.1 in /usr/local/lib/python3.7/site-packages (from internal-lib5>=0.13.2->internal-lib2==6.0.1->-r requirements.txt (line 29)) (4.3)
Requirement already satisfied: internal-lib1>=0.4.7 in /usr/local/lib/python3.7/site-packages (from internal-lib5>=0.13.2->internal-lib2==6.0.1->-r requirements.txt (line 29)) (0.4.10)

Same loop with -v flag (pip install -r requirements.txt --use-feature=2020-resolver -v):

…
Requirement already satisfied: six>=1.4.1 in environment/lib/python3.6/site-packages (from cryptography==3.0->-r requirements.txt (line 4)) (1.14.0)
1 location(s) to search for versions of six:
* https://self-hosted-devpi/+simple/six/
Fetching project page and analyzing links: https://self-hosted-devpi/+simple/six/
Getting page https://self-hosted-devpi/+simple/six/
Looking up "https://self-hosted-devpi/+simple/six/" in the cache
Request header has "max_age" as 0, cache bypassed
https://devpi.yougov.net:443 "GET /+simple/six/ HTTP/1.1" 200 16235
Updating cache with response from "https://self-hosted-devpi/+simple/six/"
  Found link https://self-hosted-devpi/%2Bf/306/39c035cdb2353/six-1.15.0.tar.gz#sha256=30639c035cdb23534cd4aa2dd52c3bf48f06e5f4a941509c8bafd8ce11080259 (from https://self-hosted-devpi/+simple/six/) (requires-python:>=2.7, !=3.0.*, !=3.1.*, !=3.2.*), version: 1.15.0
  …
  Found link https://self-hosted-devpi/%2Bf/14f/d1ed3dd0e1a46/six-0.9.0.tar.gz#sha256=14fd1ed3dd0e1a46cc53b8fc890b5a3b11737515aeb7f42c3af9f38e8d8975d7 (from https://self-hosted-devpi/+simple/six/), version: 0.9.0
Given no hashes to check 12 links for project 'six': discarding no candidates
Requirement already satisfied: internal-lib1>=0.4.7 in environment/lib/python3.6/site-packages (from internal-lib5>=0.13.2->internal-lib2==6.0.1->-r requirements.txt (line 29)) (0.4.10)
1 location(s) to search for versions of internal-lib1:
* https://self-hosted-devpi/+simple/internal-lib1/
Fetching project page and analyzing links: https://self-hosted-devpi/+simple/internal-lib1/
Getting page https://self-hosted-devpi/+simple/internal-lib1/
Looking up "https://self-hosted-devpi/+simple/internal-lib1/" in the cache
Request header has "max_age" as 0, cache bypassed
https://devpi.yougov.net:443 "GET /+simple/internal-lib1/ HTTP/1.1" 200 13870
Updating cache with response from "https://self-hosted-devpi/+simple/internal-lib1/"
  Found link https://self-hosted-devpi/%2Bf/674/b54a83be61d7e/internal-lib1-0.4.4-py2.py3-none-any.whl#sha256=674b54a83be61d7e5488edf33ac4846d4eb19b54eb662b37a1a3dcbe146d3f71 (from https://self-hosted-devpi/+simple/internal-lib1/), version: 0.4.4
  …
  Found link https://self-hosted-devpi/%2Bf/5db/e47a27ce7d29e/internal-lib1-0.3.1.tar.gz#sha256=5dbe47a27ce7d29ee0321b9252f4d00232a17042ad160861f2fc449928328d53 (from https://self-hosted-devpi/+simple/internal-lib1/), version: 0.3.1
Given no hashes to check 8 links for project 'internal-lib1': discarding no candidates
Requirement already satisfied: internal-lib3<5dev,>=4.0.1 in environment/lib/python3.6/site-packages (from internal-lib5>=0.13.2->internal-lib2==6.0.1->-r requirements.txt (line 29)) (4.3)
1 location(s) to search for versions of internal-lib1:
* https://self-hosted-devpi/+simple/internal-lib1/
Fetching project page and analyzing links: https://self-hosted-devpi/+simple/internal-lib1/
Getting page https://self-hosted-devpi/+simple/internal-lib1/
Looking up "https://self-hosted-devpi/+simple/internal-lib1/" in the cache
Request header has "max_age" as 0, cache bypassed
https://devpi.yougov.net:443 "GET /+simple/internal-lib1/ HTTP/1.1" 200 9900
Updating cache with response from "https://self-hosted-devpi/+simple/internal-lib1/"
  Found link https://self-hosted-devpi/%2Bf/ad9/b1e001e1b51f2/internal-lib3-4.1.tar.gz#sha256=ad9b1e001e1b51f2305275717e451e6d8b81b54fa621030284b7c49f11a8ace4 (from https://self-hosted-devpi/+simple/internal-lib1/), version: 4.1
  …
  Found link https://self-hosted-devpi/%2Bf/11c/81413452f331c/internal-lib3-3.3.tar.gz#sha256=11c81413452f331cc75e09c0b1c424d1574a1f9d7c2c63d70cc60e66871839aa (from https://self-hosted-devpi/+simple/internal-lib1/), version: 3.3
Given no hashes to check 9 links for project 'internal-lib1': discarding no candidates
1 location(s) to search for versions of marshmallow:
* https://self-hosted-devpi/+simple/marshmallow/
Fetching project page and analyzing links: https://self-hosted-devpi/+simple/marshmallow/
Getting page https://self-hosted-devpi/+simple/marshmallow/
Looking up "https://self-hosted-devpi/+simple/marshmallow/" in the cache
Request header has "max_age" as 0, cache bypassed
https://devpi.yougov.net:443 "GET /+simple/marshmallow/ HTTP/1.1" 200 70169
Updating cache with response from "https://self-hosted-devpi/+simple/marshmallow/"
  Found link https://self-hosted-devpi/%2Bf/a2a/5eefb4b75a3b4/marshmallow-3.7.1.tar.gz#sha256=a2a5eefb4b75a3b43f05be1cca0b6686adf56af7465c3ca629e5ad8d1e1fe13d (from https://self-hosted-devpi/+simple/marshmallow/) (requires-python:>=3.5), version: 3.7.1
  …
  Found link https://devpi.yougov.net/root/pypi/%2Bf/b2a/6f1df3c475cd9/marshmallow-0.1.0-py2.py3-none-any.whl#sha256=b2a6f1df3c475cd9c1ae29d9ef46ab1f24110bf7721d4e91c8ee90e2eaf604e8 (from https://devpi.yougov.net/+simple/marshmallow/), version: 0.1.0
Given no hashes to check 0 links for project 'marshmallow': discarding no candidates
Collecting marshmallow-oneofschema<3.0.0,>=2.0.0.post0
  Created temporary directory: /private/var/folders/22/_y1ydpt97v70hqfj5f7wvy740000gp/T/pip-unpack-q1p0op6g
  Looking up "https://self-hosted-devpi/%2Bf/296/73bcc415f4a20/marshmallow_oneofschema-2.0.1-py2.py3-none-any.whl" in the cache
  Current age based on date: 2114
  Ignoring unknown cache-control directive: immutable
  Freshness lifetime from max-age: 365000000
  The response is "fresh", returning cached response
  365000000 > 2114
  Using cached https://self-hosted-devpi/%2Bf/296/73bcc415f4a20/marshmallow_oneofschema-2.0.1-py2.py3-none-any.whl (5.7 kB)
  Added marshmallow-oneofschema<3.0.0,>=2.0.0.post0 from https://self-hosted-devpi/%2Bf/296/73bcc415f4a20/marshmallow_oneofschema-2.0.1-py2.py3-none-any.whl#sha256=29673bcc415f4a2098e273bf3c8229bc1a33e91378885eac90b06943bd626110 (from internal-lib4==0.15.0->-r requirements.txt (line 30)) to build tracker '/private/var/folders/22/_y1ydpt97v70hqfj5f7wvy740000gp/T/pip-req-tracker-ojo8i3cq'
  Removed marshmallow-oneofschema<3.0.0,>=2.0.0.post0 from https://self-hosted-devpi/%2Bf/296/73bcc415f4a20/marshmallow_oneofschema-2.0.1-py2.py3-none-any.whl#sha256=29673bcc415f4a2098e273bf3c8229bc1a33e91378885eac90b06943bd626110 (from internal-lib4==0.15.0->-r requirements.txt (line 30)) from build tracker '/private/var/folders/22/_y1ydpt97v70hqfj5f7wvy740000gp/T/pip-req-tracker-ojo8i3cq'
1 location(s) to search for versions of marshmallow:
* https://self-hosted-devpi/+simple/marshmallow/
Fetching project page and analyzing links: https://self-hosted-devpi/+simple/marshmallow/
Getting page https://self-hosted-devpi/+simple/marshmallow/
Looking up "https://self-hosted-devpi/+simple/marshmallow/" in the cache
Request header has "max_age" as 0, cache bypassed
https://devpi.yougov.net:443 "GET /+simple/marshmallow/ HTTP/1.1" 200 70169
Updating cache with response from "https://self-hosted-devpi/+simple/marshmallow/"
  Found link https://self-hosted-devpi/%2Bf/a2a/5eefb4b75a3b4/marshmallow-3.7.1.tar.gz#sha256=a2a5eefb4b75a3b43f05be1cca0b6686adf56af7465c3ca629e5ad8d1e1fe13d (from https://self-hosted-devpi/+simple/marshmallow/) (requires-python:>=3.5), version: 3.7.1
Given no hashes to check 0 links for project 'marshmallow': discarding no candidates
Collecting internal-lib5>=0.13.2
  Created temporary directory: /private/var/folders/22/_y1ydpt97v70hqfj5f7wvy740000gp/T/pip-unpack-ffx_i25r
  Looking up "https://self-hosted-devpi/%2Bf/f5d/75b9548e314c7/internal-lib5-0.13.3-py2.py3-none-any.whl" in the cache
  Current age based on date: 2114
  Ignoring unknown cache-control directive: immutable
  Freshness lifetime from max-age: 365000000
  The response is "fresh", returning cached response
  365000000 > 2114
  Using cached https://self-hosted-devpi/%2Bf/f5d/75b9548e314c7/internal-lib5-0.13.3-py2.py3-none-any.whl (13 kB)
  Added internal-lib5>=0.13.2 from https://self-hosted-devpi/%2Bf/f5d/75b9548e314c7/internal-lib5-0.13.3-py2.py3-none-any.whl#sha256=f5d75b9548e314c7a47301cae5daecb1c1167165f3f249dbef3779d81d5a7875 (from internal-lib2==6.0.1->-r requirements.txt (line 29)) to build tracker '/private/var/folders/22/_y1ydpt97v70hqfj5f7wvy740000gp/T/pip-req-tracker-ojo8i3cq'
  Removed internal-lib5>=0.13.2 from https://self-hosted-devpi/%2Bf/f5d/75b9548e314c7/internal-lib5-0.13.3-py2.py3-none-any.whl#sha256=f5d75b9548e314c7a47301cae5daecb1c1167165f3f249dbef3779d81d5a7875 (from internal-lib2==6.0.1->-r requirements.txt (line 29)) from build tracker '/private/var/folders/22/_y1ydpt97v70hqfj5f7wvy740000gp/T/pip-req-tracker-ojo8i3cq'
Requirement already satisfied: six>=1.4.1 in environment/lib/python3.6/site-packages (from cryptography==3.0->-r requirements.txt (line 4)) (1.14.0)
… [looped]

On the other hand minimal reproducible example finds conflict quickly:

ERROR: Cannot install marshmallow==3.0.0b12, internal-lib4 0.15.0, marshmallow-oneofschema 2.0.1, marshmallow==3.0.0b12, internal-lib4 0.15.0 and internal-lib4 because these package versions have conflicting dependencies.

The conflict is caused by:
    The user requested marshmallow==3.0.0b12
    internal-lib4 0.15.0 depends on marshmallow<=3.0.0rc6 and >=3.0.0b12
    marshmallow-oneofschema 2.0.1 depends on marshmallow<4.0.0 and >=3.0.0rc6
    The user requested marshmallow==3.0.0b12
    internal-lib4 0.15.0 depends on marshmallow<=3.0.0rc6 and >=3.0.0b12
    marshmallow-oneofschema 2.0.0.post0 depends on marshmallow<4.0.0 and >=3.0.0rc6

To fix this you could try to:
1. loosen the range of package versions you've specified
2. remove package versions to allow pip attempt to solve the dependency conflict

ERROR: ResolutionImpossible: for help visit https://pip.pypa.io/en/latest/user_guide/#fixing-conflicting-dependencies

Minimal requirements with conflict:

cryptography==3.0
internal-lib2==6.0.1
internal-lib4==0.15.0
marshmallow==3.0.0b12

Where:

By the way, thank you for your work on pip!

m-aciek commented 4 years ago

May be duplicate of https://github.com/pypa/pip/issues/8683.