astral-sh / uv

An extremely fast Python package and project manager, written in Rust.
https://docs.astral.sh/uv
Apache License 2.0
25.89k stars 755 forks source link

uv installs wrong (cached) version when specifying a git+ dependency with hash #6974

Closed MacLake closed 1 month ago

MacLake commented 2 months ago

I use uv 0.4.3 with Python 3.11 on Manjaro Linux.

I need to install updated versions of a package from Github. I have a dependency in my pyproject.toml:

dependencies = [
    …
    "djangocms-blog @ git+https://github.com/fsbraun/djangocms-blog.git",  # The blog application for django CMS
    …
]

uv writes the correct version hash to the requirement files:

djangocms-blog @ git+https://github.com/fsbraun/djangocms-blog.git@6b5b5f0fd716a6a3df07c32ab1897aaa34a2e325

Before I had other versions installed. As it is a package in development where the author fixes a bug, the version number defined in the source doesn’t change in every commit, but the hash does for sure. E. g. the one before is 29a799ea11dd9206bf88ff71ef61effd1692d9e6.

As I noticed from an error message, the newest version was not installed with uv pip install -r dev-requirements.txt, as the message refers to content in line 64, which still mentioned content instead of contents.

I also tried to uninstall and reinstall the package:

$ uv pip uninstall djangocms-blog
Uninstalled 1 package in 12ms                                                                                                                                                               
 - djangocms-blog==2.0.0rc1 (from git+https://github.com/fsbraun/djangocms-blog.git@6b5b5f0fd716a6a3df07c32ab1897aaa34a2e325)                                                               
$ uv pip install 'djangocms-blog @ git+https://github.com/fsbraun/djangocms-blog.git@6b5b5f0fd716a6a3df07c32ab1897aaa34a2e325'  
Resolved 35 packages in 72ms                                                                                                                                                                
Installed 1 package in 16ms                                                                                                                                                                 
 + djangocms-blog==2.0.0rc1 (from git+https://github.com/fsbraun/djangocms-blog.git@6b5b5f0fd716a6a3df07c32ab1897aaa34a2e325)                                                               
$ uv pip uninstall djangocms-blog
Uninstalled 1 package in 10ms                                                                                                                                                               
 - djangocms-blog==2.0.0rc1 (from git+https://github.com/fsbraun/djangocms-blog.git@6b5b5f0fd716a6a3df07c32ab1897aaa34a2e325)                                                               
$ pip install 'djangocms-blog @ git+https://github.com/fsbraun/djangocms-blog.git@6b5b5f0fd716a6a3df07c32ab1897aaa34a2e325' 

After uv pip uninstall djangocms-blog or pip uninstall djangocms-blog the corresponding directory is deleted, so uv apparently installs an old cached version.

With pip I get the correct version.

There are other version mismatch issues, which seem to me similar, but not the same, so I wanted to tell about this specific problem encounter.

charliermarsh commented 2 months ago

I'm having a little trouble following, but can you try passing the --upgrade flag?

MacLake commented 2 months ago

Well, to summarize it: uv pip install 'djangocms-blog @ git+https://github.com/fsbraun/djangocms-blog.git@6b5b5f0fd716a6a3df07c32ab1897aaa34a2e325' installs the wrong version on my system, even when the package is not installed right before carrying out this instruction and even though I provide the hash. I suspect it uses a cached version I had installed earlier.

Adding -U or --upgrade doesn’t help, it even shows a strange behaviour: When I first applied it, uv upgraded other packages, django and packaging and didn’t even mention the package I want to install:

uv pip install -U 'djangocms-blog @ git+https://github.com/fsbraun/djangocms-blog.git@6b5b5f0fd716a6a3df07c32ab1897aaa34a2e325'                                                                            
Resolved 34 packages in 7.84s                                                                                                      
Prepared 2 packages in 2.78s                                                                                                       
Uninstalled 2 packages in 174ms                                                                                                    
Installed 2 packages in 134ms                                                                                                      
 - django==5.0.8                                                                                                                   
 + django==5.0.9                                                                                                                   
 - packaging==21.3                                                                                                                 
 + packaging==24.1

In a second try, the output looks as expected, but still the wrong version is being installed.

uv pip install --upgrade 'djangocms-blog @ git+https://github.com/fsbraun/djangocms-blog.git@6b5b5f0fd716a6a3df07c32ab1897aaa34a2e325'
Resolved 34 packages in 1.40s
Prepared 1 package in 7ms
Installed 1 package in 4ms
 + djangocms-blog==2.0.0rc1 (from git+https://github.com/fsbraun/djangocms-blog.git@6b5b5f0fd716a6a3df07c32ab1897aaa34a2e325
charliermarsh commented 2 months ago

...django and packaging and didn’t even mention the package I want to install:

This could be completely correct if the package is already installed at the correct version. I can't know without --verbose output.

charliermarsh commented 2 months ago

Do you mind providing a clear sequence of commands that demonstrate the behavior you're seeing from a clean state? (I.e., running uv cache clean, rm -rf .venv, etc.)

charliermarsh commented 2 months ago

(I would love to help and fix any issues here.)

maparent commented 2 months ago

Not sure it's exactly the same issue, but it is related, so adding it here.

Here is a simple pyproject.toml, on macOS:

[project]
name = "test_uv"
version = "0.0.1"
requires-python = ">=3.11;<3.12"
dependencies = [
  "tensorflow-text @ https://github.com/sun1638650145/Libraries-and-Extensions-for-TensorFlow-for-Apple-Silicon/releases/download/v2.17/tensorflow_text-2.17.0-cp311-cp311-macosx_11_0_arm64.whl#sha256=f0a7225a1242f06e4d206235f56025a0f70fb9a2e3ecb85bd5ba686269e74c16 ; sys_platform == 'darwin'",
  "tensorflow-text; sys_platform == 'linux'",
]

[build-system]
requires = ["hatchling"]
build-backend = "hatchling.build"

[tool.hatch.metadata]
allow-direct-references = true

an uv sync on this will fail with this message:

Resolved 41 packages in 352ms
error: Failed to prepare distributions
  Caused by: Failed to fetch wheel: tensorflow-text @ https://github.com/sun1638650145/Libraries-and-Extensions-for-TensorFlow-for-Apple-Silicon/releases/download/v2.17/tensorflow_text-2.17.0-cp311-cp311-macosx_11_0_arm64.whl#sha256=f0a7225a1242f06e4d206235f56025a0f70fb9a2e3ecb85bd5ba686269e74c16
  Caused by: Hash mismatch for `tensorflow-text @ https://github.com/sun1638650145/Libraries-and-Extensions-for-TensorFlow-for-Apple-Silicon/releases/download/v2.17/tensorflow_text-2.17.0-cp311-cp311-macosx_11_0_arm64.whl#sha256=f0a7225a1242f06e4d206235f56025a0f70fb9a2e3ecb85bd5ba686269e74c16`

Expected:
  sha256:ff3e9a8e19256e184a0e0997b189fa4a73bb00da74a109cef542e48073d99142

Computed:
  sha256:f0a7225a1242f06e4d206235f56025a0f70fb9a2e3ecb85bd5ba686269e74c16

Note that the computed is exactly as requested.

More important: Removing line 7, "tensorflow-text; sys_platform == 'linux'", allows uv to sync successfully. The linux line should never be activated on macos; but somehow it tells uv to fetch the hash for the linux version of the wheel and create a wrong expectation.

I hope this is helpful.

charliermarsh commented 2 months ago

Thanks @maparent. Looks like a real bug but separate from the above; going to move into its own issue.

MacLake commented 2 months ago

Thanks, @charliermarsh. After cleaning the cache with uv cache clean uv pip install behaves again as expected, i. e. installing the package with git+ using a hash installs the correct version. So the bug was triggered by the state of the cache. I have saved a copy of the cache directory, so if there is something I should look for, let me know. Otherwise I’m going to delete it soon, since it’s 16 GB big.

Wehn reproducing the bug, I had deleted all other virtualenvs of this project, just in case. So for trying to reproduce this bug, one could first install an older version like

uv pip install 'djangocms-blog @ git+https://github.com/fsbraun/djangocms-blog.git@29a799ea11dd9206bf88ff71ef61effd1692d9e6'

Then verify the version by looking at djangocms_blog/migrations/0044_copy_plugins.py line 64

Uninstalling the package should not be necessary, but like this you can be sure that uv or pip use the correct virtualenv:

uv pip uninstall djangocms-blog

or shell

pip uninstall djangocms-blog

Check that the package directory in the virtualenv is empty. The install the newer version:

uv pip install 'djangocms-blog @ git+https://github.com/fsbraun/djangocms-blog.git@6b5b5f0fd716a6a3df07c32ab1897aaa34a2e325'

Check if the mentioned file has changed and corresponds to the desired version. A colleague of mine could not reproduce the bug, and after cleaning the cache I cannot either.

So I don’t know if it makes sense to chase this reason for bug or jut live with it that sometimes uv may not install the correct version. In many cases you might not even notice it. i only realized it because the error message didn’t match the code (content vs. contents).

charliermarsh commented 1 month ago

Ok, going to close out due to lack of reproduction, but let me know if you see it consistently!