astral-sh / uv

An extremely fast Python package installer and resolver, written in Rust.
https://astral.sh/
Apache License 2.0
15.5k stars 456 forks source link

Namespace package's `__init__.py` is corrupted when multiple libraries in it are installed #4831

Closed ods closed 2 weeks ago

ods commented 3 weeks ago

Given:

Form time to time (very rarely) we get a failed pipeline due to a syntax error in namespace/__init__.py. Investigation showed that it has the content from one library (of larger size) partially overwritten with the content from another library (of smaller size), leading to invalid code.

charliermarsh commented 3 weeks ago

"Partially overwritten" is strange -- like the content is a mix of two different files...?

ods commented 3 weeks ago

The content of namespace/__init__.py file in one of the linraries is:

"""Common libraries for ..........."""

__import__("pkg_resources").declare_namespace(__name__)
__version__ = "1.11.0"

the content of it in another library:

# TODO Switch to PEP-420 implicit namespace packages after moving all
# `..........` packages to `pkgutil`.
# About staged migration and interoperability tables:
# https://github.com/pypa/sample-namespace-packages/pull/22/files

from pkgutil import extend_path

__path__ = extend_path(__path__, __name__)

and the resulting content I see in the container after install:

"""Common libraries for ..........."""

__import__("pkg_resources").declare_namespace(__name__)
__version__ = "1.11.0"
ged migration and interoperability tables:
# https://github.com/pypa/sample-namespace-packages/pull/22/files

from pkgutil import extend_path

__path__ = extend_path(__path__, __name__)

"Partially overwritten" is my interpretation of what's going on.

konstin commented 3 weeks ago

That looks bad indeed! On what platform are you on, are you setting --link-mode and could you check the verbose log (-v) for messages with attempting to copy files as a fallback in it?

ods commented 3 weeks ago

On what platform are you on

In docker, image is based on official python:3.12.4-slim, linux/x86_64

are you setting --link-mode

No. The actually used command is:

RUN --mount=from=uv,source=/uv,target=/bin/uv \
    --mount=type=cache,target=${HOME}/.cache,uid=${UID},gid=${GID} \
    uv pip install --system --prefix ${HOME}/.local -r requirements.txt --require-hashes --no-deps

could you check the verbose log (-v) for messages with attempting to copy files as a fallback in it?

Not sure how to achieve this. We switched back to pip after discovering the problem, so uv is not used in pipeline anymore. I'll try to find a way to reproduce the problem locally.