Python discovery prefers python3 over python3.12 when UV_PYTHON=python3.12

[hynek]

This is something that happens with uv 0.5.0 and 0.5.1 but not uv 0.4.30.

I've run into this in Ubuntu Noble containers where for some reason my fat build container carries a /usr/bin/python3 but my slim prod container does not – but it has /usr/bin/python3.12. Both are the same system Python 3.12.

Before 0.5.0, if UV_PYTHON is set to python3.12, uv sync and uv venv would already say Using CPython 3.12.3 interpreter at: /usr/bin/python3 but the symlink from the venv would go to /usr/bin/python3.12. Now it goes to /usr/bin/python3 and the venv is broken in prod.

The workaround is setting UV_PYTHON=/usr/bin/python3.12

[charliermarsh]

Yeah this came up in Discord yesterday. I'm not sure what changed exactly. My guess is that we used to fully resolve those symlinks, but now we follow the Python standard library and only use sys._base_executable for the virtual environment Python.

[zanieb]

Yeah I can't think of anything but #8481 that would have changed here.

[zanieb]

I guess the main question is why sys._base_executable is wrong here? This should be "broken" with python -m venv too right?

[zanieb]

I can fix this specific case by preferring a versioned executable over an unversioned one when a version is requested, but it's just more complexity in the discovery rules and won't help if there's a symlink in another directory.

[charliermarsh]

@hynek -- Do you have any sort of minimal repro we can use to look at how the Python setup works?

[hynek]

Sure:

# syntax=docker/dockerfile:1.9
FROM ubuntu:noble AS build
SHELL ["sh", "-exc"]

RUN <<EOT
apt-get update
apt-get install -qyy \
    -o APT::Install-Recommends=false \
    -o APT::Install-Suggests=false \
    python3-setuptools \
    python3.12
EOT

COPY --from=ghcr.io/astral-sh/uv:latest /uv /usr/local/bin/uv

ENV UV_LINK_MODE=copy \
    UV_PYTHON_DOWNLOADS=never \
    UV_PYTHON=python3.12

RUN <<EOT
uv venv /app
ls -l /app/bin/python
EOT

##########################################################################

FROM ubuntu:noble
SHELL ["sh", "-exc"]

RUN <<EOT
apt-get update
apt-get install -qyy \
    -o APT::Install-Recommends=false \
    -o APT::Install-Suggests=false \
    python3.12
EOT

COPY --from=build /app /app

RUN <<EOT
ls -l /app/bin/python
/app/bin/python -V
EOT

FWIW it can also be triggered by installing python3 instead of python3-setuptools but that's unlikely to happen in a Dockerfile. This is how I ran into the problem.

[zanieb]

Should be fixed by #9066

[zanieb]

I would still like to understand why the base executable path is wrong.

[bluss]

Isn't it the case here that there are two docker images. The first python environment has /usr/bin/python3. Then the venv is copied into a different image where /usr/bin/python3 doesn't exist. Uv doesn't do much wrong, the venv was transported between two "incompatible" python environments?

[charliermarsh]

Yeah I don't know that the base executable path is "wrong". It just depends on what Python itself reports.

[zanieb]

It still seems weird that a different base executable would be reported depending on if python or python3 is queried.