astral-sh / uv

An extremely fast Python package and project manager, written in Rust.
https://docs.astral.sh/uv
Apache License 2.0
26.01k stars 762 forks source link

Transitive PROJECT_ROOT is expanded in requirements.txt #3750

Open patrick-kidger opened 5 months ago

patrick-kidger commented 5 months ago

Reproduction: Given the following directories:

one/pyproject.toml
two/pyproject.toml
three/pyproject.toml

with contents:

# one/pyproject.toml
[project]
name = "one"
version = "0.0.1"
dependencies = ["two @ file://${PROJECT_ROOT}/../two"]
# two/pyproject.toml
[project]
name = "two"
version = "0.0.1"
dependencies = ["three @ file://${PROJECT_ROOT}/../three"]
# three/pyproject.toml
[project]
name = "three"
version = "0.0.1"
dependencies = []

then running

one> uv pip compile pyproject.toml -o requirements.txt

will (correctly) produce a requirements.txt with the contents

# This file was autogenerated by uv via the following command:
#    uv pip compile pyproject.toml -o requirements.txt
three @ file://${PROJECT_ROOT}/../three
    # via two
two @ file://${PROJECT_ROOT}/../two
    # via one (pyproject.toml)

however a second invocation of the same command now produces a requirements.txt with the contents

# This file was autogenerated by uv via the following command:
#    uv pip compile pyproject.toml -o requirements.txt
three @ file:///Absolute/Path/To/three
    # via two
two @ file://${PROJECT_ROOT}/../two
    # via one (pyproject.toml)

which has had PROJECT_ROOT expanded in the transitive dependency!

Expected behaviour is for PROJECT_ROOT to be preserved in requirements.txt, and for uv pip compile to be idempotent.

Calling uv cache clean will result in the next invocation of uv pip compile being correct, so this seems to be a caching issue. This is using uv version 0.1.45 on macOS.

charliermarsh commented 5 months ago

Def a bug, thanks.

charliermarsh commented 5 months ago

\cc @konstin - could this be related to any of your URL work?

konstin commented 5 months ago

I've confirmed that this bug exists since 0.1.25 when env vars in pyproject.toml was introduced. The problem is complex to solve: When we cache requirements, we store them as string with their env var resolved, losing the verbatim representation on the url. I tried to change the cache representation of requirements to the extended from (0cb98f419c01935da19b5a82c48d2ec1f2159407), but this clashes with the build hooks and toml parsing trying to return metadata through serde too, but having requirements as string because they come from python or toml respectively, with Metadata23 being used both with the stringly case and in the cache.