search

astral-sh / uv

An extremely fast Python package and project manager, written in Rust.
https://docs.astral.sh/uv
Apache License 2.0
26.65k stars 773 forks source link

Performance with large git repos #1737

Open sbidoul opened 9 months ago

sbidoul commented 9 months ago

Hi,

I found a situation where uv is significantly slower than pip: installing from a large git repo.

Reproducer: uv -v pip install "odoo @ git+https://github.com/odoo/odoo@17.0" (warning: this requires ~6GB disk space and a ~4GB download).

After the download (observed by monitoring network activity), there seems to be a pause of several minutes where it is unclear what uv is doing, even with -v or RUST_LOG=trace.

charliermarsh commented 9 months ago

Thanks!

charliermarsh commented 9 months ago

@zanieb - This may be solved by changing the default to use the Git CLI.

serozhenka commented 8 months ago

I also experience a strange pause for around 7 seconds (out of 17 for the overall install) when trying to install a private git package via a git tag and not a commit ref. As @sbidoul mentioned nothing indicates what's going on even when enabling additional verbosity.

zanieb commented 8 months ago

@serozhenka is this on the latest version?

serozhenka commented 8 months ago

@zanieb it was 0.1.6 at the time of writing, but I have just checked on 0.1.8 and it's the same. Installation time dropped down by 3s on average, but that lag still exists.

upd. my bad on that, missed a single log statement, that's actually the repo subdirectory fetch itself that takes 8s. But overall it's still 1-2s slower that pip with cold cache.

sbidoul commented 8 months ago

From my part, I have noticed a solid improvement with the switch to using the git CLI.

In pip we use git clone --filter=blob:none to improve performance with large repos, but the pip git clones are transient (we only keep the resulting wheel).

So I think (I've not mesured rigorously) that with a cold cache, uv is slower than pip with large git repos, but when the git db cache is pre-populated, subsequent operations will be faster.