astral-sh / uv

An extremely fast Python package and project manager, written in Rust.
https://docs.astral.sh/uv
Apache License 2.0
19.95k stars 591 forks source link

Clarify the benefits of `uv.lock` over `requirements.txt` #6271

Open zanieb opened 3 weeks ago

zanieb commented 3 weeks ago

e.g. as discussed in Discord

Yes, but there are a couple caveats. (1) requirements.txt is not automatically used — you need to manually construct the environment and remember to do so every time you change branches. In contrast, uv run automatically ensures that uv.lock is up-to-date and that all commands are run in a consistent environment. (2) uv.lock uses cross-platform resolution by default, requirements.txt only targets a single platform (though you can use the --universal flag to generate a cross-platform file). The uv.lock format is has more information about requirements in it and is designed to be performant and auditable.

danielkelshaw commented 3 weeks ago

I was just reading over the new docs for uv (which are great by the way!) and am just wondering about the use case for the uv.lock vs. requirements.txt? I understand that the uv.lock captures the current environment, is it not also the job for the requirements.txt to capture this? If so, would it make sense to have something like:

$ uv pip compile uv.lock -o requirements.txt

in order to capture the exact state of the environment, rather than relying on the pyproject.toml for instance? Perhaps I have misunderstood - but a small section in the docs to clarify may be helpful!

zanieb commented 3 weeks ago

uv.lock resolves the dependencies declared in the pyproject.toml — it doesn't capture the current environment. Instead, the project's environment is constructed from the lockfile.

Similarly,requirements.txt can be generated from a pyproject.toml — but it also supports other sources, like a requirements.in file.

In both cases, you'd sync an environment with the lockfile. It's a matter of format. Unfortunately requirements.txt is an unspecified and complex format, which makes it hard to extend. In contrast, uv.lock has a bunch of information that lets us improve the performance of subsequent resolutions andensure that the environment is up-to-date before running commands. We also designed it to precisely track package sources for easy auditing.

astronautas commented 3 weeks ago

uv.lock resolves the dependencies declared in the pyproject.toml — it doesn't capture the current environment. Instead, the project's environment is constructed from the lockfile.

Similarly,requirements.txt can be generated from a pyproject.toml — but it also supports other sources, like a requirements.in file.

In both cases, you'd sync an environment with the lockfile. It's a matter of format. Unfortunately requirements.txt is an unspecified and complex format, which makes it hard to extend. In contrast, uv.lock has a bunch of information that lets us improve the performance of subsequent resolutions andensure that the environment is up-to-date before running commands. We also designed it to precisely track package sources for easy auditing.

Hmm, yet another lockfile format. Not a biggie though, it's just a matter of regenerating it and using uv within your CI/CD instead of pip. But still.

zanieb commented 3 weeks ago

We are engaging heavily in PEP 751 — which defines a standard for lockfiles. Once there's a standard, we plan to use it.

keatingw commented 3 weeks ago

Let me know if this should go into a separate issue, but the one missing link for me with uv.lock is an ability to install from it independent of the project. The main use case on my mind is caching dependency installs with Docker (I note the docs highlight the way to do this with the pip interface, but I can't think of a way to do so with the lockfile at present).

I'm currently just duplicating the workflow by using uv pip compile to create a requirements.txt in addition to the lockfile, but would love to avoid this since it's hard to make sure that this is actually in sync with uv.lock, rather than a new resolution for the dependencies declared. (Plus removing the duplication is a side benefit, though not a huge deal)

Thanks for the fantastic work on uv!

zanieb commented 3 weeks ago

@keatingw thank you! Extensive discussion about that at https://github.com/astral-sh/uv/issues/4028 — we should provide a way to do that soon.