Support `--user` flag from pip

[potiuk]

Currently (even in latest uv 0.1.12 that supports --python flag) it's not possible to use uv in case --user flag of pip (or PIP_USER="true" env variable is set).

With --user flag, packages are installed to a ~/.local folder, which means that the user might not have access to the python system installation as a whole, but can locally install and uninstall packages, without having venv.

This is extremely useful for cases like CI and building container images, where having a venv is an extra overhead and it is unnecessary burden.

There is an interesting (and used in Apache Airflow) case for the --user flag (this currently prevent us from using uv by us and our users also in PROD images in addition to our CI image). That is currently a blocker for https://github.com/apache/airflow/issues/37785

Why --user flag is useful and why we use it in Airflow?

Using --user flag is pretty useful when you want to have an optimized image - because such .local folders can be copied between different stage of the image (based on the same base image and prod libraries installed) and you can copy the whole .local folder between the stages after packages are installed - which means that the final stage of the image does not have to have build-essentials/ compilers installed.

You could do the same with venv if you keep it in the same folder, but that looses an essential capability of creating venv dynamically containing all the packages you have in your .local folder. When you have your --user installed packages, and you create a new venv with --system-site-packages, the packages installed in .local folder are also installed in the new venv. This does not work when you create a new venv from another venv because --system-site-packages are only the ones installed in the system.

While I can think of some creative ways (maybe I will find some) - having an equivalent of --user installation by uv pip install would be a great simplification for our case to support the users who want to use uv (and seems that there is already a need for that looking at the https://github.com/apache/airflow/issues/37785

[potiuk]

Just to answer the previous post - why mix the discussion of --user and default virtualenvs? - because, if there are no immediate plans for a "just-works" solution for the beginner, such as default virtualenv, then I would be interested in having the --user option as a not-quite-as-good solution to the same problem - even if it was only temporary.

When you build product (like the uv team does) - temporary solutions usually turn into long-term maintenance headache, because those who start using it expect it to continue working. So while you might have a wish to solve your problem, you have to understand that maintainers will have to live with those decisions for a long time.

[matthew-brett]

Ah yes I understand - and I do build product. But I do want to push back on "your problem" - it's really not my problem - but a problem that will be common for beginners.

[potiuk]

Ah yes I understand - and I do build product. But I do want to push back on "your problem" - it's really not my problem - but a problem that will be common for beginners.

So let maintainers solve it in their due time. Without pushing them, as they have likely many other things to work on wich are likely higher priority.

[zanieb]

because, if there are no immediate plans for a "just-works" solution for the beginner

These plans are immediate; we're building out next-generation workflows on top of the foundation we built for uv pip right now. These workflows are intended to abstract virtual environment management. We'll have more to announce on this front soon.

[Malix-Labs]

Currently, UV makes it impossible to use python without a .venv, because it purposely disables the --user flag, even if it would not the default, and no recommendation would be made, or even by putting additional warnings for too curious noobs

Why not allowing --user, granted it's not the default, not recommended unless in a container, and need to be done with an explicit flag?

For me, a non-noob user but new to the mess of python ecosystem, and also using standard tools such as containers instead of virtual environment, .venv is another layer, and a different workflow which is also more manual and prone to error (source .venv/bin/activate each time), which has shot me in the feet enough to say I would definitely prefer container + --user.

I was hoping Astral would unify things Yet, it doesn't for me

[zanieb]

Frankly, we don't make it impossible to use Python without a virtual environment. We allow installing into non-virtual Python environments in a myriad of ways, e.g., with --python <path> or --system. There are workarounds for user-level installs, e.g., using --target. There are ways to avoid manual activation of the environment. Additionally, we are actively building workflows for uv that do not require you to ever manually manage a virtual environment.

I understand you don't have write permissions to the system site packages in your devcontainer context and that means you need to understand a bit more to set up your environment. However, we must take into account the trade-offs for the situations of many users.

edit: I see your post was edited a bunch times while I was writing this, sorry if I've responded to something out of date.

[Malix-Labs]

• Are --python and --target available on every uv pip commands (or --python available for some commands and --target for the other, or some commands don't support either) ?

• Which flag + argument would be a 1:1 behaviour replication to --user for all uv pip use-cases ?

• you need to understand a bit more to set up your environment

  For my use-case, understanding .venv would still be inferior to using raw --user (or another 1:1 flag + command), even if I was willing to specifically understand a bit more python's .venv

• However, we must take into account the trade-offs for the situations of many users.

  I don't understand how allowing --user, while still continuing to not recommend it anywhere to make it foolproof, would be an inferior trade-off compared to not making it possible for those who want to use --user to use it (and would know they really want to use it, at that point)

• I see your post was edited a bunch times while I was writing this, sorry if I've responded to something out of date

  Yes, no problem it's still up-to-date 🙂

[zanieb]

Are --python and --target available on every uv pip commands

--python is available on all of the uv pip commands. --target is available on mutating commands like install, uninstall, and sync.

Which flag + argument would be a 1:1 behaviour replication to --user for all uv pip use-cases ?

Generally the invocation looks like this:

❯ uv pip install --target "$(python3 -m site --user-site)" fastapi
...
❯ python3 -c "import fastapi; print(fastapi.__version__)"
0.111.0

Note this was also discussed further up in this thread.

I think we'd need to add --target to the other commands if you want to use it for inspection — which I don't see an immediate problem with, nobody has asked for it yet.

or my use-case, understanding .venv would still be inferior...

Perhaps you would find this draft documentation helpful https://github.com/astral-sh/uv/pull/4433

I don't understand how allowing --user...

I think you've been quite clear that you want the --user flag — we appreciate the feedback but we'd rather focus on new ways to address this use-case. While it may not seem like much effort to add, we are ultimately responsible for maintaining this tool for years to come. Let's try not to rehash the discussions we've already had here and focus on new perspectives and use-cases.

[jbcpollak]

I've settled on "faking" a venv in my environments so I'm fine without --user but I wanted to point out this:

Generally the invocation looks like this:

❯ uv pip install --target "$(python3 -m site --user-site)" fastapi ... ❯ python3 -c "import fastapi; print(fastapi.version)" 0.111.0

Does not seem to work, at least on Windows. Some of my team are still using "bare" Python installs on windows and global installations of dependencies, but when I try the above command (adapted the path resolution to Windows), I get the error about no venv found. Is that supposed to be bypassed when using --target?

[imfing]

@jbcpollak I remember both --target and --python are needed

See my previous comment: https://github.com/astral-sh/uv/issues/2077#issuecomment-2150406001

uv pip install \
    --python $(which python) \
    --target $(python -m site --user-site) \
    -r requirements.txt

[sqrl]

This workaround doesn't seem to create a /bin sub-directory the way --user does? Testing on uv version 0.2.33 I believe.

[charliermarsh]

Which workaround are you referring to?

[sqrl]

Which workaround are you referring to?

The one mentioned in this comment: https://github.com/astral-sh/uv/issues/2077#issuecomment-2150406001

[imfing]

Which workaround are you referring to?

The one mentioned in this comment: #2077 (comment)

@sqrl use --prefix instead of --target should work better in this case:

uv pip install \
    --python $(which python) \
    --prefix $(python -m site --user-base) \
    -r requirements.txt

See https://github.com/astral-sh/uv/issues/2059#issuecomment-2227402333 and https://github.com/astral-sh/uv/pull/4085

[matthew-brett]

Hi - sorry - I scanned the preview labels - but couldn't see the proposed solution for having a default virtualenv - did I miss it? Or is it still in progress?

[Rotonen]

As this seems to lead into a lot of confusion and keeps resurrecting, here's a recipe.

Amend your own base container (concatenated as a multistage image for brevity / being a self contained demo):

FROM python:3.12-slim-bookworm as setup-user
# Always run your end software as non-root!
RUN useradd -ms /bin/bash python
USER python
WORKDIR /home/python

FROM setup-user as setup-venv
# Setup upstream standard venv
#
# XXX - as upstream venv gives you setuptools and friends, those should also
# be pinned in a separate requirements-setup.txt or such and installed in a
# separate bootstrapping phase to keep things actually well defined and
# stable as they get bumped forward when the upstream Python image gets
# updated - omitted for clarity here.
#
# The three lines below are probably what you're missing from stuff
# "just working"!
RUN python -m venv .venv
ENV VIRTUAL_ENVIRONMENT=/home/python/.venv
ENV PATH=/home/python/.venv/bin:"$PATH"

FROM setup-venv as setup-uv
# XXX - This would be defined via a requirements-uv.txt or such in real use.
RUN pip install uv==0.3.3

FROM setup-uv as use-uv
# XXX - This would be defined via a requirements.txt or such in real use.
RUN uv pip install pyfiglet==1.0.2

FROM use-uv as everything-works
# Separating ENTRYPOINT and CMD is in general a nice idea
ENTRYPOINT ["pyfiglet"]
CMD ["uv rocks!"]

Then on the downstream consumer side in your org the consumer can simply pip install or uv pip install as they please and everything will happen via that predefined venv as intended.

IMO no changes on the side of uv needed. Feature parity with all sins of the past is not the way forward.

[matthew-brett]

Just to clarify - you can read the details further up, but the use-case here is the absolute beginner, running the equivalent of uv install pygame, in their default Python environment, and without having to explain virtualenvs.

[zanieb]

Hi - sorry - I scanned the preview labels - but couldn't see the proposed solution for having a default virtualenv - did I miss it? Or is it still in progress?

We didn't introduce any preview behavior around this — we're probably not going to support a --user flag or a default virtual environment.

[Malix-Labs]

When working with devcontainers or nix , forbidding to use an explicit --user flag (and thus forcing to use a venv anyway) is a net negative

[matthew-brett]

Aha - ouch - well - I'm afraid that will make uv inaccessible to beginners - but I understand you have to optimize to some subset of users ...

[matthew-brett]

I guess the plan will be to propose uv to more advanced users who are comfortable with virtualenvs. But that will necessarily mean some division of the user-base - with some using uv and others starting, and perhaps continuing with some other tool, that does allow just-works installs without understanding virtualenvs.

[zanieb]

We very much care about supporting beginners and eliminating the need to understand virtual environments — but we don't think a global mutable environment is the way to do that.

[matthew-brett]

Aha - could you say more about other approaches you are considering?

[edmorley]

One caveat to bear in mind when skipping using a venv and instead using --user with the global Python installation: If you are using a relocated Python install, some packages doesn't correctly locate the stdlib out of the box, eg: https://github.com/unbit/uwsgi/issues/2525

Working around that requires setting PYTHONHOME explicitly, which then comes with its own set of issues.

In contrast, support for PEP-405 style venvs seems much more prevalent.

It's for this reason that I'm in the process of migrating the Heroku Python CNB to use a venv in the layer containing the app dependencies instead of a user site-packages installation.

If you are using a non-relocated Python install (such as the Python in the python:* images on Docker Hub), then this issue doesn't apply. But it's something to bear in mind wrt the "venvs aren't needed for containers" argument, since it's not always true.

[Malix-Labs]

Do that mean that the python ecosystem has some dependency on venv usage ?

[edmorley]

More that: (a) by design venvs are in a different location from system Python so tooling can't avoid handling the sys.prefix vs sys.base_prefix distinction, which sidesteps any issues with relocated Python if the packaging/tools don't do the right thing otherwise (b) in general venvs appear to be the more well-trodden path (particularly when comparing to --user, which is rarer than 100% system installs), since so many workflows/tools now use them by default (eg Poetry, Pipenv, uv) - and sticking to the well tested path tends to result in fewer surprises overall

[Malix-Labs]

1. Understandable, but isn't relocating supposed to be solved by fixing the path in /usr/bin/env too ?
2. Understandable too, are there any other language than Python having venv ? Is working in a devcontainer an unpopular workflow in python, or at least way less popular than in other languages ?

[edmorley]

Understandable, but isn't relocating supposed to be solved by fixing the path in /usr/bin/env too ?

No that doesn't help. Please see the STR in https://github.com/unbit/uwsgi/issues/2525

(we're getting a bit off-topic here, so this will be my last reply :-) )