.lock file causes conflict when installing python into a shared directory

[kwaegel]

I'm having trouble getting installing Python builds (and tools) into a directory shared by multiple users, due to a single-user owned .lock file created in UV_PYTHON_INSTALL_DIR. I'm not sure if what I am attempting is reasonable or not, but at the moment I can't tell if this is a bug or intended behaviour.

My setup is roughly following the suggestion from this prior issue.

Background: I'm trying to create a Dockerfile that's used in a CI environment. The Dockerfile itself runs all the setup commands as root, but the Jenkins bot uses the user ubuntu. I'm attempting to install everything in a shared directory /opt/uv/*, allowing either user to invoke uv python install ... or uv tool install ... commands.

Minimal Dockerfile replicating this issue:

FROM ubuntu:24.04 AS build
COPY --from=ghcr.io/astral-sh/uv:0.4.20 /uv /bin/uv
ENV UV_PYTHON_INSTALL_DIR="/opt/uv/python"
RUN uv python install 3.10
USER ubuntu
RUN uv python install 3.11

Error message:

#9 [build 4/4] RUN uv python install 3.11
#9 0.248 error: failed to create file `/opt/uv/python/.lock`
#9 0.248   Caused by: Permission denied (os error 13)
#9 ERROR: process "/bin/sh -c uv python install 3.11" did not complete successfully: exit code: 2

The situation for installing tools is analogous , with an owned .lock file created in UV_TOOL_DIR.

My current somewhat awkward workaround is to set the setgid bit on the shared folder and add umask 002 before each command, which makes the .lock file writable by all group members. The downside is that it's easy to forget to include the umask prefix on every command that needs it.

FROM ubuntu:24.04 AS build
COPY --from=ghcr.io/astral-sh/uv:0.4.20 /uv /bin/uv
ENV UV_PYTHON_INSTALL_DIR="/opt/uv/python"
RUN mkdir -p "/opt/uv" && \
    chgrp ubuntu "/opt/uv" && \
    chmod g+s "/opt/uv"
RUN umask 002 && uv python install 3.10
USER ubuntu
RUN umask 002 && uv python install 3.11

[zanieb]

Interesting. I'm not sure what we can do to fix this, but it seems worth trying to.

[kwaegel]

I'm not entirely sure what the .lock file is used for, but I had initially assumed it would work something like:

• Create the lock file to get exclusive access to the python (or tools) directory.
• Preform any add/remove operations.
• Remove the lock file.

The lock file persisting between write operations is one thing that confuses me, so maybe it's doing more than I had assumed?

[zanieb]

Oh it shouldn't persist across operations, afaik. Yes it's a file used for exclusive access, e.g.,

https://github.com/astral-sh/uv/blob/f0659e76cf215480d30d554118961c7c59ab3e2b/crates/uv-tool/src/lib.rs#L142-L145

[charliermarsh]

I think It might need to persist across operations... Can a lock holder know that's safe to delete? Other processes may be waiting for it.

[zanieb]

Yeah I think you're right. Do we need to create the file with different permissions when we're in a globally writable directory?

[kwaegel]

I'm curious why it would need to persist across operations. Wouldn't most operations be a lock/write/unlock cycle? If that was the case, the lock would only ever be created and removed by the same user. That might only be true for adding Python versions, though. Removing versions might be more problematic, but that's not something my CI images need to deal with.

Another consequence of the current state is that uv python install x.y.z fails even if x.y.z is installed, since it tries to acquire the lock before checking for an existing instillation. I can do a manual workaround like below, but it feels like this should be automatic (pyenv uses a --skip-existing flag for similar behaviour).

uv python find $(PYTHON_VERSION) || uv python install $(PYTHON_VERSION)

[charliermarsh]

The purpose of the lock is to prevent shared modification of a virtual environment across two concurrent processes. I don't think the writer can remove it after unlock -- what if another writer is waiting to access it? I actually have no idea what happens in that case.

Should we just be writing it with more open permissions?

[kwaegel]

For my purposes, open permissions (when globally writable, or when the setgid bit is enabled) would resolve the issue, since I'm doing multi-user but not concurrent writes, but I'm not sure if that's safe in the general case.