search

astral-sh / uv

An extremely fast Python package and project manager, written in Rust.
https://docs.astral.sh/uv
Apache License 2.0
19.93k stars 591 forks source link

question for feature request: is there a way to "pin" a specific dependency to the latest? #6404

Open qci-amos opened 3 weeks ago

qci-amos commented 3 weeks ago

I'm looking for a way to do the equivalent of: specify a pin of a dependency to whatever the latest currently is.

For example, say I have a requirements.txt that has something like:

a
b
botocore

but at the time of installation, would be the equivalent of:

a
b
botocore==1.35.3

(since that's the latest for botocore as of this writing). I can't just save that as a requirements.txt because botocore makes a new release like every day. A specific use case is that botocore has like 2000 releases so I don't want it to try to backtrack on it if it needs to search for compatible versions.

My main use case is to give my non-python-expert users a command they can use to to ensure they're on the very latest versions of my own code but I don't want to force them to upgrade any other dependencies unnecessarily.

zanieb commented 3 weeks ago

This doesn't exist, but it might be feasible. I'd like to hear some more use-cases from other users first though, it seems hard to get right.

BrendanJM commented 3 weeks ago

Maybe a similar issue, we've seen issues specifically with botocore and related libs where the uv resolver will always fail to resolve due to the sheer number of versions. The backtracking does not seem to be able to reach a solution, so we end up with arbitrary pinning in all of our requirements files:

# Note: These are only pinned because the dep resolver could not figure out valid versions otherwise
# There is nothing special about these versions.
boto3==1.34.131
botocore==1.34.131
s3fs>=2024.6.1

Even doing something like pinning lower bound to the ranges e.g. boto3>=1.34.131 does not seem to work, and the resolver ends up going down the rabbit hole for some adjacent transitive like aiobotocore. This is a partial requirements spec for the main package that requires this pinning (can't post full due to several internal libraries):

boto3
mercantile
numpy
packaging
pandas
psycopg[binary]
pyarrow
pydantic>=2.0.0
pyproj
pyspark>=3.4.0
python-geohash
requests
s3fs
shapely
statsmodels
tldextract
urllib3

It is surprising because it seems like if the dep resolver took a different search strategy (e.g. searching most recent versions for all first) it should be a fairly fast resolve, but because it wants to walk down versions one lib at a time, it ends up looking at hundreds of boto-related versions before failing.

zanieb commented 3 weeks ago

@BrendanJM lots of prior discussion about that, e.g., in https://github.com/astral-sh/uv/issues/1398 or https://github.com/astral-sh/uv/issues/4333 — sorry you're having trouble with it though. It's something we want to explore but it's very hard.

qci-amos commented 3 weeks ago

For what it's worth, this is an issue I've written up elsewhere, too: https://github.com/pypa/pip/issues/12028

A suggestion made there was perhaps a new version specifier, bit it seems to me it could be some kind of resolver feature, too.