Poetry doesn't respect version specifiers in Requires-Dist for dependencies

f0ff886f commented 4 years ago

[x] I am on the latest Poetry version.
[x] I have searched the issues of this repo and believe that this is not a duplicate.
[x] If an exception occurs when executing a command, I executed it again in debug mode (-vvv option).

OS version and name: Windows 10 2004
Poetry version: 1.0.10
Link of a Gist with the contents of your pyproject.toml file: https://gist.github.com/f0ff886f/3178ffdf080e67a5fdfa3fcbd5503d31

Issue

poetry add rich@latest

...

[SolverProblemError]
The current project's Python requirement (^3.6) is not compatible with some of the required packages
Python requirement:
  - dataclasses requires Python >=3.6, <3.7

Because no versions of rich match >7.0.0,<8.0.0
 and rich (7.0.0) depends on dataclasses (>=0.7,<0.8), rich (>=7.0.0,<8.0.0) requires dataclasses (>=
0.7,<0.8).
Because dataclasses (0.7) requires Python >=3.6, <3.7
 and no versions of dataclasses match >0.7,<0.8, dataclasses is forbidden.
Thus, rich is forbidden.

This is related to but doesn't seem to be directly addressed by the many "Python requirement (^3.6) is not compatible with some of the required packages" issues. Closest issue is https://github.com/python-poetry/poetry/issues/2840 and the solution https://github.com/python-poetry/poetry/issues/1930#issuecomment-653906544 doesn't apply here.

My library is expressing it is compatible with Python ^3.6, which I understand as >=3.6.0, < 4.0.0.

It requires the https://github.com/willmcgugan/rich library which is compatible also with Python ^3.6. The issues starts with Rich's dependency on dataclasses which started to throw a wrench in many people's plans it seems by declaring its Python requirement as >=3.6, <3.7 (^3.7 has dataclasses built in).

Now Rich declares its dependency like so: Requires-Dist: dataclasses (>=0.7,<0.8); python_version >= "3.6" and python_version < "3.7"

When dataclasses is added as a dependency with python_version ~3.6 directly to this project, it resolves dataclasses just fine (which I understand is just shorthand for >=3.6, <3.7).

But now, when Rich tries to do the same, and I depend on Rich, everything breaks and cannot continue.

The posted solution says to use python_version specifiers, but I cannot do that for Rich, but Rich itself does that and Poetry seems to ignore it.

The workaround is to manually specify rich = "^7.0.0" in the [tool.poetry.dependencies] and then something like poetry update works just fine.

f0ff886f commented 4 years ago

Note, this same dependency check will continue to fail if you do something completely unrelated to rich: poetry remove httpx will then complain about rich's dependencies, I'm guessing it tries to re-evaluate the entire dependency graph.

abn commented 4 years ago

@f0ff886f can you try with the latest pre-release please?

Also, fwiw, I am unable to reproduce this.


$ cat pyproject.toml 
[tool.poetry]
name = "foo"
version = "0.16.0"
description = ""
authors = ["Author <Author@Email.Com>"]

[tool.poetry.dependencies]
python = "^3.6"
httpx = "^0.13.3"
colorama = "^0.4.3"
click = "^7.0"
rich = "^3.1.0"
importlib-metadata = "^1.7.0"

[tool.poetry.dev-dependencies]
black = "^19.10b0"
flake8 = "^3.7.9"

[tool.poetry.scripts]
ovat = 'client.cli:main'

[build-system]
requires = ["poetry>=0.12"]
build-backend = "poetry.masonry.api"
$ poetry@1.0.10 add rich@latest
Using version ^7.0.0 for rich

Updating dependencies
Resolving dependencies... (0.2s)

Writing lock file

Package operations: 0 installs, 1 update, 1 removal

  - Updating rich (3.4.1 -> 7.0.0)
  - Removing pprintpp (0.4.0)
$ poetry@1.0.10 remove httpx
Updating dependencies
Resolving dependencies... (0.1s)

Writing lock file

Package operations: 0 installs, 0 updates, 14 removals

  - Removing certifi (2020.6.20)
  - Removing chardet (3.0.4)
  - Removing contextvars (2.4)
  - Removing h11 (0.9.0)
  - Removing h2 (3.2.0)
  - Removing hpack (3.0.0)
  - Removing hstspreload (2020.9.22)
  - Removing httpcore (0.9.1)
  - Removing httpx (0.13.3)
  - Removing hyperframe (5.2.0)
  - Removing idna (2.10)
  - Removing immutables (0.14)
  - Removing rfc3986 (1.4.0)
  - Removing sniffio (1.1.0)

f0ff886f commented 4 years ago

This just got a ton weirder after updating to 1.1.0b3. Things looked promising to start!

❯ poetry remove httpx
Updating dependencies
Resolving dependencies...

Writing lock file

Package operations: 0 installs, 0 updates, 10 removals

  • Removing certifi (2020.6.20)
  • Removing chardet (3.0.4)
  • Removing contextvars (2.4)
  • Removing h11 (0.9.0)
  • Removing httpcore (0.10.2)
  • Removing httpx (0.14.3)
  • Removing idna (2.10)
  • Removing immutables (0.14)
  • Removing rfc3986 (1.4.0)
  • Removing sniffio (1.1.0)

client on  master [$!?] is � v0.16.1 took 5s
❯ poetry remove rich
Updating dependencies
Resolving dependencies...

Writing lock file

Package operations: 0 installs, 0 updates, 4 removals

  • Removing commonmark (0.9.1)
  • Removing pygments (2.7.1)
  • Removing rich (7.0.0)
  • Removing typing-extensions (3.7.4.3)

client on  master [$!?] is � v0.16.1 took 5s

And re-applying things:

❯ poetry add rich
Using version ^7.0.0 for rich

Updating dependencies
Resolving dependencies...

Writing lock file

Package operations: 4 installs, 0 updates, 0 removals

  • Installing commonmark (0.9.1)
  • Installing pygments (2.7.1)
  • Installing typing-extensions (3.7.4.3)
  • Installing rich (7.0.0)

client on  master [$!?] is � v0.16.1 took 9s
❯ poetry add httpx
Using version ^0.15.0 for httpx

Updating dependencies
Resolving dependencies...

  SolverProblemError

  The current project's Python requirement (>=3.6,<4.0) is not compatible with some of the required pack
ages Python requirement:
    - dataclasses requires Python >=3.6, <3.7, so it will not be satisfied for Python >=3.7,<4.0

  Because no versions of rich match >7.0.0,<8.0.0
   and rich (7.0.0) depends on dataclasses (>=0.7,<0.8), rich (>=7.0.0,<8.0.0) requires dataclasses (>=0
.7,<0.8).
  Because dataclasses (0.7) requires Python >=3.6, <3.7
   and no versions of dataclasses match >0.7,<0.8, dataclasses is forbidden.
  Thus, rich is forbidden.
  So, because client depends on rich (^7.0.0), version solving failed.

  at ~\.poetry\lib\poetry\puzzle\solver.py:241 in _solve
      237│             packages = result.packages
      238│         except OverrideNeeded as e:
      239│             return self.solve_in_compatibility_mode(e.overrides, use_latest=use_latest)
      240│         except SolveFailure as e:
    → 241│             raise SolverProblemError(e)
      242│
      243│         results = dict(
      244│             depth_first_search(
      245│                 PackageNode(self._package, packages), aggregate_package_nodes

  • Check your dependencies Python requirement: The Python requirement can be specified via the `python`
 or `markers` properties

    For dataclasses, a possible solution would be to set the `python` property to ">=3.6,<3.7"

    https://python-poetry.org/docs/dependency-specification/#python-restricted-dependencies,
    https://python-poetry.org/docs/dependency-specification/#using-environment-markers

What the heck 🤔 I don't know how to explain this.

f0ff886f commented 4 years ago

I can reproduce this by replacing Rich with another temporary library that expresses the same dependency on dataclasses ^0.7 python_version >=3.6, < 3.7.

I have noticed that things like black and pydantic require dataclasses >= 0.6 python_version < 3.7 ... perhaps the problem is deeper into dataclasses 0.7.

Yet still, if I manually add the dependencies to the pyproject.toml, poetry update completes without an issue.

willmcgugan commented 4 years ago

Is it not suspicious that it appears to be asking for Rich greater than 7.0.0, when rich is currently = 7.0.0?

abn commented 4 years ago

@f0ff886f we did identify a bug in 1.1.0b3 (#2950) and we will be releasing a fix for this soon. Try using 1.1.0b2 for now.

Although, as I mentioned above I was unable to reproduce the issue with 1.0.10. If there is something specific that I am missing, please do point that out.

f0ff886f commented 4 years ago

I had a friend try the same project with 1.0.10 and it fails the exact same way as I reported originally. I don't know what to hint that may differ for us... but still, great news that 1.1.0b3 has identified the regression! I'll try to find out how to install 1.1.0b2 with the poetry installer and give it a shot.

The friend was able to at least poetry install with 1.1.0b3 instead of 1.0.10, so there's a definitive difference in behaviour on our end.

@willmcgugan I never really realized that it could be a problem. Let me change the version specifier and see what happens.

edit: nope, that didn't help. rich = ">=7.0.0, <8.0.0" didn't change anything with the error.

f0ff886f commented 4 years ago

@abn downgrading to 1.1.0b2 solved the dependency resolution! BUT... now I hit the exact same problem as in https://github.com/python-poetry/poetry/issues/2911 so that also does not work (every installation of a dependency fails).

So 1.1.0b3 doesn't respect the dependency's python_version, and 1.1.0b2 causes errors on every dependency installation.

I'll just wait for 1.1.0b5... 👍 I really appreciate the help.

f0ff886f commented 4 years ago

Hello, thank you for the work in 1.1.0rc1! I can confirm this bug is gone and I have a fully working and perfectly working Poetry!

github-actions[bot] commented 8 months ago

This issue has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.

python-poetry / poetry

Poetry doesn't respect version specifiers in Requires-Dist for dependencies #2946

Issue