astral-sh / rye

a Hassle-Free Python Experience
https://rye.astral.sh
MIT License
13.84k stars 466 forks source link

PROJECT_ROOT conveted to absolute path in lockfile #451

Open sksat opened 1 year ago

sksat commented 1 year ago

Steps to Reproduce

$ cd /tmp
$ rye init hoge --build-system pdm
$ rye init fuga
$ rye add --path ../fuga fuga
Added fuga @ file:///${PROJECT_ROOT}/../fuga as regular dependency
$ rye lock
$ cat requirements.lock | grep fuga

Expected Result

../fuga

Actual Result

fuga @ file:///tmp/hoge/../fuga

Version Info

rye 0.13.0
commit: 0.13.0 (8e011231e 2023-08-29)
platform: linux (x86_64)
self-python: cpython@3.11
symlink support: true

Stacktrace

No response

CharlesChen0823 commented 1 year ago

I think the behavior of current is same as pdm

carderne commented 1 year ago

The solution is to set up a workspace, then all local dependencies will be added to requirements.lock as simply -e file:package_name.

But then you still can't install the path package (fuga in your case) in the dependent package (hoge), as rye/pip-tools will hit a conflict resolving the plain file:package_name and the file:///tmp... that you have.

What I've done is installed the path package as an optional dependency rye add --optional local --path ../fuga fuga, which does nothing to requirements.lock, and then using something like this in my Dockerfiles:

COPY hoge/pyproject.toml requirements.lock ./
RUN sed -i '/^-e file:/d' requirements.lock
RUN pip install '.[local]' --constraint requirements.lock
np-kyokyo commented 10 months ago

Seriously, this. Otherwise, it can't be shared.

mitsuhiko commented 10 months ago

Unfortunately the choices here are limited because the lock file "standard" is relatively restricted. Rye already prefers relative paths in lock files where possible.

np-kyokyo commented 10 months ago

Thank. I wanted to install the openapi_client generated by openapi positioned in a subdirectory as a package. This issue seem to be resolved by properly setting the packageName option in the openapi generator (e.g., --additional-properties=packageName=src.openapi.generated.openapi_client). I will write more if there is anything else.

roy-ht commented 8 months ago

I got some results with uv.

In requirements.lock file, wrote local path dependency like xxx @ file:///abc/def is related to your build backend.

here is used pyproject.toml:

[project]
name = "temp"
version = "0.1.0"
description = "Add your description here"
authors = [
    { name = "Hiroyuki Tanaka", email = "aflc0x@gmail.com" }
]
dependencies = [
    "core @ file:///${PROJECT_ROOT}/../core",
]
readme = "README.md"
requires-python = ">= 3.8"
classifiers = ["Private :: Do Not Upload"]

[build-system]
requires = ["setuptools"]
build-backend = "setuptools.build_meta"

# [build-system]
# requires = ["pdm-backend"]
# build-backend = "pdm.backend"

# [build-system]
# requires = ["hatchling"]
# build-backend = "hatchling.build"

[tool.rye]
managed = true
dev-dependencies = []

[tool.hatch.metadata]
allow-direct-references = true

[tool.hatch.build.targets.wheel]
packages = ["src/temp"]

I checked the generated lock file of three backends and got a result:

hatch: failed to build: "ValueError: Unknown context field PROJECT_ROOT" pdm: absolute path in lockfile setuptools: "file:///${PROJECT_ROOT}/../core" in lockfile

here is a lock file generated by using setuptools backend:

# generated by rye
# use `rye lock` or `rye sync` to update this lockfile
#
# last locked with the following flags:
#   pre: false
#   features: []
#   all-features: false
#   with-sources: false

-e file:.
core @ file:///${PROJECT_ROOT}/../core
    # via temp
wheel==0.42.0
    # via core

My guess

uv seems to use local path info in wheels Requires-Dist section of .dist-info/METADATA file. if I built wheel using pdm, Requires-Dict section is absolute path, and wheel using setuptools is "core@ file:///${PROJECT_ROOT}/../core".

Possible solution

If uv could convert the path to the relative, like "file:///${PROJECT_ROOT}" in its post-process if dependencies in pyproject.toml has "file:///${PROJECT_ROOT}" like path.