Closed charliermarsh closed 1 month ago
I added a test (that shows the failure) in https://github.com/astral-sh/uv/pull/4648.
It would be great if the universal resolver would support situations like this. For a software package at work we would like to support a very broad range of Python versions (Python 3.7 - Python 3.12) and a common problem is the numerical Python stack (numpy, pandas) the software depends on. Quite often there is no numpy version which would offer upstream wheels for all Python versions we want to support. Having support for a forked resolution of numpy in such cases would be great. Maybe the fact of having Python version specific requirements/constraints could be a marker when forking is wished for.
To be clear, the resolver does support divergent requirements, for example this resolves without issue:
requests==2.32.3 ; python_version >= '3.12'
requests==2.32.0 ; python_version < '3.12'
It's the blanket requests
that's causing problems (for details related to the internal implementation).
I have some difficulties with numpy
(macOS, uv 0.2.18). With a requirements.in
numpy >=1.26; python_version>="3.9"
numpy <1.26; python_version<"3.9"
I cannot univeral-lock for Python >= 3.8:
$ uv pip compile -p 3.8 --universal requirements.in
× No solution found when resolving dependencies:
╰─▶ Because only the following versions of numpy{python_version >= '3.9'} are available:
numpy{python_version >= '3.9'}<=1.26.0
numpy{python_version >= '3.9'}==1.26.1
numpy{python_version >= '3.9'}==1.26.2
numpy{python_version >= '3.9'}==1.26.3
numpy{python_version >= '3.9'}==1.26.4
numpy{python_version >= '3.9'}==2.0.0
and the requested Python version (3.8) does not satisfy Python>=3.9, we can conclude that any of:
numpy{python_version >= '3.9'}>=1.26.0,<1.26.2
numpy{python_version >= '3.9'}>1.26.2,<1.26.3
numpy{python_version >= '3.9'}>1.26.3,<1.26.4
numpy{python_version >= '3.9'}>1.26.4,<2.0.0
numpy{python_version >= '3.9'}>2.0.0
are incompatible.
And because the requested Python version (3.8) does not satisfy Python>=3.9 and you require
numpy{python_version >= '3.9'}>=1.26, we can conclude that the requirements are unsatisfiable.
Thanks, that should work but likely a separate issue.
It works in the general case, e.g.:
anyio >= 3 ; python_version >= '3.9'
anyio < 3 ; python_version < '3.9'
My guess is that the issue in your case is that numpy >=1.26
doesn't support Python 3.8, and we're still trying to lock for your requires-python
in that branch (i.e., we're still trying to enforce that every dependency supports Python 3.8 and later, instead of narrowing the requirement in that branch). I will create a separate issue.
Conceptually, could we model this as:
requests ; sys_platform != 'darwin' and sys_platform != 'linux'
requests==2.32.3 ; sys_platform == 'darwin'
requests==2.32.0 ; sys_platform == 'linux'
@BurntSushi?
The way forking works currently is that it looks for conflicting dependency specifications with non-overlapping markers. If conflicting dependency specifications are found but have overlapping markers, then a fork doesn't occur. So I think that if you start with something like:
requests
requests==2.32.3 ; python_version >= '3.12'
requests==2.32.0 ; python_version < '3.12'
Then because requests
has no markers, it is considered overlapping with any other dependency specification on requests
. So in this case, I believe no fork occurs at all. (I haven't actually confirmed that though.)
The suggestion of treating the requests
specification as if its markers are non-overlapping by construction with each other specification is interesting. I'm not sure we can do that in the general case, but if we have some information like "requests
was from requirements.in
/pyproject.toml
" and "the others are extra constraints added," then maybe it makes sense to massage the specification from requirements.in
to be non-overlapping with the constraints given? Are there cases where we would do this and it would be undesirable?
Another option, perhaps, is to improve the error message and require the user to change their dependency specification to add the non-overlapping marker. But I'm not sure how well that works with specifying constraints.
Given:
Running
uv pip compile requirements.txt --universal
fails with:This seems a bit silly, but it's maybe not quite as silly if you view it as constraints. Imagine your
requirements.txt
is:And then your
constraints.txt
is:This would also fail.