Support `--user` flag from pip

[potiuk]

Currently (even in latest uv 0.1.12 that supports --python flag) it's not possible to use uv in case --user flag of pip (or PIP_USER="true" env variable is set).

With --user flag, packages are installed to a ~/.local folder, which means that the user might not have access to the python system installation as a whole, but can locally install and uninstall packages, without having venv.

This is extremely useful for cases like CI and building container images, where having a venv is an extra overhead and it is unnecessary burden.

There is an interesting (and used in Apache Airflow) case for the --user flag (this currently prevent us from using uv by us and our users also in PROD images in addition to our CI image). That is currently a blocker for https://github.com/apache/airflow/issues/37785

Why --user flag is useful and why we use it in Airflow?

Using --user flag is pretty useful when you want to have an optimized image - because such .local folders can be copied between different stage of the image (based on the same base image and prod libraries installed) and you can copy the whole .local folder between the stages after packages are installed - which means that the final stage of the image does not have to have build-essentials/ compilers installed.

You could do the same with venv if you keep it in the same folder, but that looses an essential capability of creating venv dynamically containing all the packages you have in your .local folder. When you have your --user installed packages, and you create a new venv with --system-site-packages, the packages installed in .local folder are also installed in the new venv. This does not work when you create a new venv from another venv because --system-site-packages are only the ones installed in the system.

While I can think of some creative ways (maybe I will find some) - having an equivalent of --user installation by uv pip install would be a great simplification for our case to support the users who want to use uv (and seems that there is already a need for that looking at the https://github.com/apache/airflow/issues/37785

[charliermarsh]

Thanks! I really appreciate the clear write-up and motivation here. (I'm hoping to get to this today, it's been requested a few times.)

[potiuk]

Just to take off the pressure a bit. I figured out how to get rid of the --user flag (still have a problem to fix) https://github.com/apache/airflow/pull/37796 - I attempted it quite some time ago and failed but this time I got a brilliant idea - I simply deleted the ~/.local folder and created a new ~/.local venv, added VIRTUAL_ENV=~/.local env var and added ~/.local/bin to PATH and .... voila ... It works 100% compatible with --user flag it seems - all the cases I had (including pip -m venv --system-site-packages) work like a charm.

So ... It seems I do not need it that much any more .... I will make Airflow PROD image also uv friendly now :) (128s instead of 280s is the gain I have :). Not bad - another 55% improvement.

There is a small difference though with venv when using python -m venv and uv venv. With uv you do not have --system-site-packages option yet. That might be the more useful of the two.

After that experience I have a new thought. The --user flag has a few bad side effects (that's why I am glad to finally get rid of it). One of the problems is that you cannot run --user and --editable any more with pip. So maybe - just maybe (?), rather than implementing --user flag - documenting how to get an equivalent by creating ~/.local venv is a better option ?

[charliermarsh]

@potiuk - At the very least I can add --system-site-packages, that's really easy. I'm somewhat undecided on --user... Gonna sleep on it.

[potiuk]

@potiuk - At the very least I can add --system-site-packages, that's really easy. I'm somewhat undecided on --user... Gonna sleep on it.

Yep. That would be a good start :). I think pip maintainers would gladly drop that one (--user) . So my proposal for the uv team is think very deeply on whatever you add. You are now way faster on adding things and responding to new requests - comparing to pip - but mainly because you do not have the whole baggage that pip accumulated over the years and where adding every single small change will make some loud part of your users unhappy. I think it should be a very conscious decision to add somethig that you will have to maintain in the future.

[charliermarsh]

Yeah I strongly agree with this. Very good callout. It's nice to add things that unlock user workflows, but there are some areas where we actively want to change user behavior, and we need to hold firm on some of those lines.

[charliermarsh]

Can I ask why you use --system-site-packages here?

[potiuk]

Yes. In Airflow we have something called PythonVirtualenvOperator. Airflow has operators (4000+ of them 😱 ) that can do tasks related to some "stuff" to do - some of them are specific ("GoogleCloudJobOperator,ApacheSparkJobOperator,CreateEKSClusterOperator) but we have a number of a few generic onesBashOperator,PythonOperator,ExternalPythonOperator` - runs python code on prepared virtualenv. All of those are unaffected, but PythonVirtualenvOperator is special - because what it does is:

It creates a new virtualenv based on requirements, python version. One of the options (only valid if your python version matches the system version) is system-site-packages - when True, the new venv will have airflow and all airflow packages pre-installed.

This is super important, because we serialize arguments that we pass to the operator, and de-serialze return value from it, as interface between airflow and the "new venv" method executed. And in order to serialize some objects, the target venv should have the "airflow + often other packages" installed. Moreover the python code to be executed (basically a methpd) can do some local imports - expecting airflow or other packages to be installed.

The --system-site-packages in this case is best, because you do not have to explicitly specify which airflow version or which other packages you have to have installed. You can specify extra dependencies to add - but having the base same as the Airflow execution environment is often necessary.

Here howto: https://airflow.apache.org/docs/apache-airflow/stable/howto/operator/python.html#pythonvirtualenvoperator Here Python API: https://airflow.apache.org/docs/apache-airflow/stable/_api/airflow/operators/python/index.html#airflow.operators.python.PythonVirtualenvOperator

[potiuk]

Under the hood, the operator does:

def _generate_virtualenv_cmd(tmp_dir: str, python_bin: str, system_site_packages: bool) -> list[str]:
    cmd = [sys.executable, "-m", "virtualenv", tmp_dir]
    if system_site_packages:
        cmd.append("--system-site-packages")
    if python_bin is not None:
        cmd.append(f"--python={python_bin}")
    return cmd

[potiuk]

The way how it works now (just built and tested a new image):

With system-site packages:

airflow@a249d829a411:/opt/airflow$ python -m virtualenv --system-site-packages ~/.aaaa
created virtual environment CPython3.10.13.final.0-64 in 422ms
  creator CPython3Posix(dest=/home/airflow/.aaaa, clear=False, no_vcs_ignore=False, global=True)
  seeder FromAppData(download=False, pip=bundle, setuptools=bundle, wheel=bundle, via=copy, app_data_dir=/home/airflow/.local/share/virtualenv)
    added seed packages: pip==24.0, setuptools==69.1.0, wheel==0.42.0
  activators BashActivator,CShellActivator,FishActivator,NushellActivator,PowerShellActivator,PythonActivator
airflow@a249d829a411:/opt/airflow$ ~/.aaaa/bin/python -m pip freeze
adal==1.2.7
adlfs==2024.2.0
aiobotocore==2.12.0
aiofiles==23.2.1
aiohttp==3.9.3
aioitertools==0.11.0
aiosignal==1.3.1
alembic==1.13.1
amqp==5.2.0
anyio==4.3.0
apache-airflow @ file:///docker-context-files/apache_airflow-2.9.0.dev0-py3-none-any.whl
apache-airflow-providers-amazon @ file:///docker-context-files/apache_airflow_providers_amazon-8.18.0.dev0-py3-none-any.whl
apache-airflow-providers-celery @ file:///docker-context-files/apache_airflow_providers_celery-3.6.0.dev0-py3-none-any.whl
apache-airflow-providers-cncf-kubernetes @ file:///docker-context-files/apache_airflow_providers_cncf_kubernetes-8.0.0.dev0-py3-none-any.whl
apache-airflow-providers-common-io @ file:///docker-context-files/apache_airflow_providers_common_io-1.3.0.dev0-py3-none-any.whl
.....  AND 300+ other packages.

Without:

airflow@a249d829a411:/opt/airflow$ python -m virtualenv  ~/.bbbb
created virtual environment CPython3.10.13.final.0-64 in 113ms
  creator CPython3Posix(dest=/home/airflow/.bbbb, clear=False, no_vcs_ignore=False, global=False)
  seeder FromAppData(download=False, pip=bundle, setuptools=bundle, wheel=bundle, via=copy, app_data_dir=/home/airflow/.local/share/virtualenv)
    added seed packages: pip==24.0, setuptools==69.1.0, wheel==0.42.0
  activators BashActivator,CShellActivator,FishActivator,NushellActivator,PowerShellActivator,PythonActivator
airflow@a249d829a411:/opt/airflow$ ~/.bbbb/bin/python -m pip freeze
airflow@a249d829a411:/opt/airflow$.

[charliermarsh]

The --user flag has a few bad side effects (that's why I am glad to finally get rid of it). One of the problems is that you cannot run --user and --editable any more with pip. So maybe - just maybe (?), rather than implementing --user flag - documenting how to get an equivalent by creating ~/.local venv is a better option ?

@potiuk -- I'm trying to make a decision on whether to support --user. Do you have any references you could share for these side effects in pip? 🙏

[potiuk]

@potiuk -- I'm trying to make a decision on whether to support --user. Do you have any references you could share for these side effects in pip? 🙏

The starting point I have is this:

And I think there were lots of related discussions:

Here is one: https://github.com/pypa/pip/issues/6375 And here is the PR that added the message above: https://github.com/pypa/pip/pull/6370

Looong discussions. But basically the gist of it is that --editable and --user do not play well together in the light of what has been agreed for PEP 517.

Personally I consider --user feature of pip as should be depreacated and besides the historical logic and the behaviour of making the .local venv automatically available for any venv, it has very little use, if you consider that you could create .local as a venv. I think it was introduced in https://peps.python.org/pep-0370/ and there were some discussions on deprecating it,

But I am not fully aware about some history behind .local and even less about future of it to make some "good" advice on it - treat it with a pinch of salt.

We luckily got rid the --user flag (and PIP_USER variable) even if pip version of our container images and Airflow 2.9.0 one (in a month or so) will not have it any more.

[imfing]

@potiuk Thank you for this issue, and thanks for the inputs you provided

Some of my thoughts regarding the comment above:

The starting point I have is this:

using the --system-site-packages while creating the virtual environment should make this error message gone. If we take a step back, --user wouldn't be quite useful in this case since there's already an virtualenv.

And I think there were lots of related discussions. Here is one: https://github.com/pypa/pip/issues/6375

I skimmed through it, and it looks like the issue was mainly about --editable install breaking when there's pyproject.toml file. It doesn't seem like --user flag had anything to do with it, since, according to the original issue: both pip install --user -e . and pip install -e . fail.

From what I understand, PEP 517 itself discusses allowing projects to specify different build systems through pyproject.toml. The build process should be independent from where the packages are installed. Not sure how that would impact user site installation.

Personally I consider --user feature of pip as should be depreacated ... there were some discussions on deprecating it,

--user was introduced over a decade ago in PEP 370. I also saw this thread about deprecating it. some replies were not in favor of deprecation, for example:

Given that we're working towards making user site-packages the default
install location in pip, removing that feature at the interpreter
level would be rather counterproductive :)

Virtual environments are a useful tool if you're a professional
developer, but for a lot of folks just doing ad hoc personal
scripting, they're more complexity than is needed, and the simple "my
packages" vs "the system's package" split is a better option. (It's
also rather useful for bootstrapping tools like "pipsi" - "pip install
--user pipsi", then "pipsi install" the other commands you want access
to).

Cheers,
Nick.

If we do a quick search on GitHub for the --user flag: https://github.com/search?q=%22pip+install+--user%22&type=code&p=1 we'll see it's still widely used

Though I haven't used --user much lately, thanks to venv and Docker containers, it's still a popular choice for many Python users for installing packages without needing admin privileges in many environments other than CI

---

Putting --user flag in uv would bring some of the complexity accumulated in pip over the years.

PR #2352 was my preliminary attempt to fit it into the uv pip implementation without introducing too much complexity as the script does the heavy lifting here.

I haven't looked into every combination of how --user flag would play with other pip flags, but it would take some effort to make it compatible with the behavior of pip for sure.

Ultimately, it's up to the uv team to decide whether to support it.

[danielhollas]

Virtual environments are a useful tool if you're a professional developer, but for a lot of folks just doing ad hoc personal scripting, they're more complexity than is needed, and the simple "my packages" vs "the system's package" split is a better option.

This is an excellent quote, thanks @imfing for noting this. As somebody coming from the scientific Python world, I was meaning to write something along these lines in this thread as well.

Another useful framing of this is the distinction between a python project versus a single-file script. For the latter, having a separate venv for each individual script would not only be annoying but also error prone, as you'd need to be constantly activating / deactivating environments, when often all you need is stdlib and numpy/scipy.

We also have a rather odball use case for this -- we have a very unholy setup where we provide a system python installation in a container image with a lot of pre-installed packages, but we allow users to install additional packages to ~/.local, which is backed by a docker volume so that it is persisted when the container exits. We can't have separate venvs since we have a lot of packages installed in the system environment (e.g. the whole jupyter stack) and they can't be duplicated.

Small side note: If this feature is accepted, I think providing --user ahould also automatically enable --strict. That's what pip does as well, makes a lot of sense in terms of making sure the user knows if they broke themselves.

(all that said, it's brilliant that uv defaults to requiring a venv, that's already a huge change with respect to pip, and forced me personally to start using them, where previously I've used conda + pip, not a great combo. I never quite got myself to understand the various subtleties - venv x virtualenv etc.)

[zanieb]

the simple "my packages" vs "the system's package" split is a better option

Note this perspective is something that we could solve with new workflows rather than matching the existing --user interface.

It's worth keeping in mind throughout this discussion that we will be building new workflows on top of the fundamental tooling we've built for pip-compatibility. If we can avoid implementing and maintaining something with complex compatibility concerns we can focus on innovation and comprehensive solutions.

[potiuk]

It's worth keeping in mind throughout this discussion that we will be building new workflows on top of the fundamental tooling we've built for pip-compatibility. If we can avoid implementing and maintaining something with complex compatibility concerns we can focus on innovation and comprehensive solutions.

This is also my concern - I think (and Airflow use-case proves that) implementing something like that and having users rely on it should be conscious decision, because it will stay with uv once the versioning and compatibility rules are set (which is inevitable) and the more compatibility concerns and use cases you have the more it slows you down the moment you hit 1.0.0 version (assuming 1.0.0 and some kind of semver-ish approach will be adopted).

And --user is a bit a can of worms when you open it, to be honest - especially that uv also allows to create venvs and have some default reliance on venv being effectively required. Which is eventually a good thing even if few years ago I went into a long (and in hindsight unnecessary and far too heated) debate with pip maintainers over their push to venv being primary and the only valid way of installing python packages (especially in the context of docker containers). It even resulted with this medium article: https://potiuk.com/to-virtualenv-or-not-to-virtualenv-for-docker-this-is-the-question-6f980d753b46

I re-read that article again (it's really funny to read such an article few years later and confront your current views with the views of few-years-younger yourself). Suprisingly (or maybe now), I tend to agree with that old-myself in many of the things I wrote there, but with uv opening not only new chapter, but a new not-yet-written book, maybe there is a good solution that I can propose here?

I think PEP-370 proposal was good, and the reason described above (some people do not care about venv and they want to do things quickly) is very valid (recognising that non-power users want to just get-on with their install is also an argument in my article actually).

But the discovery I made in https://github.com/apache/airflow/issues/37785 that I can simply create a venv in ~/.local made me think that maybe we can connect venv creation and non-power users approach and both eat cake and have it?

What happens currently when you start uv pip install without a venv ? You get an error telling you tha tno virtualenv is detected and possibly that you can use --python to point to a python installation.

Is it good for non power users ? Not at all. It's confusing for first time users who have no knowledge about venv and they have no idea where their python is. Do we want to teach them that? I am not sure. We do not want to teach people all the details about venv etc. if all they want is to instal some package and use it, and even less if they are running uv pip install in their Dockerfile. What we want instead - we want them to USE venv. Even if they have no idea they are doing it.

But wait - we already have a way to create venvs (fast) in uv that we fully control here.... Why don't we ..... create venv for such user? Why don't we ..... create it in ~/.local (or equivalent place on other system) automatically if it is not there (and we can even check if ~/.local/bin is in the path and suggest the user to add it to make use of installed entrypoints. That's also nicely compatible with PEP-370 (which is more about using the ~/.local installation place rather thatn creating it.

That would be my proposal in short:

• forget the --user flag
• fallback to creating / using ~/.local venv when no virtualenv and no --python flag is used

[matthew-brett]

Just to say - I too would be very interested in a mechanism to support beginners who do not yet know about virtualenvs. I have taught a lot of students to use Python for data analysis, including data science. In practice these beginners (and there are lot of them) will not need to (consciously) make virtualenvs for a while - because the standard Python data stack is pretty stable, and doesn't generate many dependency conflicts. I'd estimate these beginners can make six months or more of progress before they need to start thinking about making and switching virtualenvs. I think it will be a significant barrier if they have to learn virtualenvs and their structure before they install their first Python package. So I would love a solution like @potiuk's - where the user can ask for, or even just be given, a default virtualenv, into which they install their packages. Of course the --user install provides something like this, but I agree with @potiuk - a default virtualenv is preferable - and in fact I already found myself making that suggestion over at this homebrew discussion. I really care about this, so I am very happy to help with anything I can to test or even build (when term finishes in a couple of weeks).

[matthew-brett]

In case it's helpful - the use-case I have in mind is - I believe - very common - and that is the beginner starting a class or an online tutorial. For example, consider this tutorial:

https://realpython.com/pygame-a-primer/

It has the instructions, right at the top, of:

pip install pygame

Or - for my students, I suggest:

pip install jupyterlab numpy scipy matplotlib

In both cases, this gets the beginner going with early Python work. But - with various installation methods - including uv pip as stands, outside Conda, this gives e.g.:

$ uv pip install pygame
error: Failed to locate a virtualenv or Conda environment (checked: `VIRTUAL_ENV`, `CONDA_PREFIX`, and `.venv`). Run `uv venv` to create a virtualenv.

Of course this isn't going to worry an experienced user, but it will be confusing to a beginner, who does not know what a virtualenv is - and - for a beginner, who doesn't understand Python installation structure, the virtualenv is a relatively advanced subject.

So - it would be very good to have some default set up, such that the beginner would not face this learning curve immediately, but face it later, when their needs and experience have expanded.

I'm posting here because I have, up until now - suggested my students do e.g. pip install --user pygame as a workaround, but I can see the arguments against doing that.

[stefanv]

Maybe the topic, of whether or not to add a --user flag, obfuscates the main concern here:

New users, as @matthew-brett mentions, are often given "pip install x" instructions. This should ideally just work, and definitely not raise obscure errors that confuse new users who know nothing about virtual envs.

Ideally, I'd imagine something like: if in venv, install there, otherwise install into a local default venv. The local default venv has access to system packages, but shadows them, so that the user can upgrade packages.

This would make for a simple first user experience, compatible with all tutorials out there, without impacting sophisticated users who can set up their own venvs.

[potiuk]

This should ideally just work,

Yep. My proposal is that it will just install things in .local (or equivalent on other systems) venv created and used automatically by uv in this case. Bonus point to print warning if .local/bin is not on PATH

[Malix-Labs]

I would definitely love that

When you're already working in a containerized development environment (nix shell / devcontainer …), it makes perfect sense (and even should be the default). Also, --system doesn't work in immutable OSes (and important tools such as Codespaces)

[matthew-brett]

Yep. My proposal is that it will just install things in .local (or equivalent on other systems) venv created and used automatically by uv in this case. Bonus point to print warning if .local/bin is not on PATH

I wonder though, whether anyone would worry about previous --user installs either being overwritten, or appearing unexpectedly in the uv environment? And where would the packages go? In .local/site-packages or somewhere else?

[Malix-Labs]

As of today, it's still not possible to use uv in immutable (where --system couldn't work) containers (such as Codespaces) without an useless (in that case) additional venv

This issue would be very appreciated for Codespaces users

[imfing]

As of today, it's still not possible to use uv in immutable (where --system couldn't work) containers (such as Codespaces) without an useless venv

@Malix-off You could use --target as a workaround, and point to /home/codespace/.local/lib/python3.10/site-packages/. For example, in a blank workspace with Python 3.10:

/workspaces/codespaces-blank $ uv pip install --target /home/codespace/.local/lib/python3.10/site-packages/ -p /home/codespace/.python/current/bin/python3.10 flask
Resolved 7 packages in 55ms
Downloaded 5 packages in 38ms
Installed 5 packages in 8ms
 + blinker==1.8.2
 + click==8.1.7
 + flask==3.0.3
 + itsdangerous==2.2.0
 + werkzeug==3.0.3

/workspaces/codespaces-blank $ python3.10 -c "import flask; print(flask.__version__)"
3.0.3

[zanieb]

Could you share a bit more about why the "useless" virtual environment matters? It seems like the overhead of creating it should be negligible.

[eswolinsky3241]

@zanieb For us, we use containerized cloud development environments because our users aren't technically advanced enough to understand virtual environments. So we use development containers to offer the same project isolation. This works perfectly with pip, where we can just have them clone the repo, run pip install -r requirements.txt and have the packages installed to the $HOME/.local directory. Having them run commands they don't understand to create a venv and activate it every time they log in (or maintaining scripts to automate this) is something we'd rather not have to deal with to get the performance improvement of uv.

[jbcpollak]

As a work around, currently for our devcontainer we have (effectively):

"postCreateCommand": "uv venv && uv pip install -r ${PROJDIR}/requirements.txt",

and in our production Dockerfile:

ENV VIRTUAL_ENV=/app/.venv

# Copy files and other setup here

RUN uv venv && \
    uv pip install -r ./requirements.txt && \
    uv pip install -e .

I would prefer if inside the containers we didn't need to have a virtual environment at all though.

[imfing]

@jbcpollak @eswolinsky3241 to avoid creating the virtual environment, this may be a workaround:

FROM python:3.10-slim
RUN pip install --no-cache-dir uv

RUN uv pip install \
    --python $(which python) \
    --target $(python -m site --user-site) \
    -r requirements.txt

it should behave the same as using the --user flag by installing the packages into user site (my requirements.txt contains only requests):

docker run --rm -it --entrypoint python 217c1ad27560 -m pip list
Package            Version
------------------ --------
certifi            2024.6.2
charset-normalizer 3.3.2
idna               3.7
pip                23.0.1
requests           2.32.3
setuptools         65.5.1
urllib3            2.2.1
uv                 0.2.6
wheel              0.41.2

[blueraft]

Another workaround to using uv in codespaces.

[matthew-brett]

Just a meta-point here - as I said above - I teach a lot of beginners - and to beginners, using a virtualenv is extremely confusing - because you really have to understand the details of a Python installation - which is far too much for a beginner coder to take on. I mean the type of coder who is still learning for-loops. And of course, we're very often teaching Python to beginner coders. Unfortunately - but inevitably - installation tools tend to be written by very experienced developers, to whom virtualenvs are no barrier. But I can assure you from long long experience that virtualenvs will be a barrier to beginners. Just for example, my third year students, who already know Python and Git and Python data science libraries - got pretty edgy when faced with pyenv in a class - despite some explanation - and even for them, it would have been a few hours of explanation and practice to get them into a state where they were ready to experiment with virtualenvs.

On the other hand - it seems to me this is very easily soluble by having a default virtualenv - where the user would need to understand virtualenvs only at the point that they really need them - when they get version conflicts or start serious development.

[Malix-Labs]

Nice workarounds,

Do you know if there is a current workaround (--user unavailable yet)

• Without "remoteUser": "root"
• Working on immutable (/ rootless) systems (such as codespaces…) AND on mutable systems (such as regular Docker devcontainers)
• No hardcoded target (i.e. hardcoding the "/home/codespace/.local/lib/python3.10/site-packages/" string)

?

[imfing]

@Malix-off check out my other comment for Dockerfile: https://github.com/astral-sh/uv/issues/2077#issuecomment-2150232739 root is not needed, and usually not recommended.

The hardcoded target user site-packages directory can be easily retrieved via python -m site --user-site:

uv pip install \
    --python $(which python) \
    --target $(python -m site --user-site) \
    -r requirements.txt

It works for both codespaces as I commented in https://github.com/astral-sh/uv/issues/2077#issuecomment-2150008781 and docker containers https://github.com/astral-sh/uv/issues/2077#issuecomment-2150232739

[zanieb]

@matthew-brett fwiw we're trying to address those problems holistically in new CLI interfaces instead of just tacking on a workaround to the uv pip interface.

[matthew-brett]

@matthew-brett fwiw we're trying to address those problems holistically in new CLI interfaces instead of just tacking on a workaround to the uv pip interface.

Nice - thanks - is there a good place to look for that?

[zanieb]

All of the new interface work is labeled as preview. We'll be publicizing more user-facing content in the next couple months.

[Malix-Labs]

Indeed @imfing, it seems that your workaround is the best until this issue is solved !

[Malix-Labs]

I am testing @imfing's workaround:

I get different results between uv pip list and pip list after running

uv pip install \
    --python $(which python) \
    --target $(python -m site --user-site) \
    fastapi

Do you know if that's normal? How should uv pip compile, uv pip sync, and uv pip freeze be used?

[imfing]

@Malix-off this is expected as uv pip list doesn't look for packages other than virtual environment or system site-packages (if --system is specified). However, both python and pip would take user site packages directory into account (PEP 370)

uv pip compile and uv pip sync support --target flag, but uv pip freeze doesn't. It is also worth mentioning that the default Python system packages could be ignored if --target is provided since it makes uv pip to only search for packages under the target directory.

[Malix-Labs]

Okay, thanks for that clarification!

We will still be forced to use venv for now, until --user is available

[charliermarsh]

To clarify, there's a good chance that we'll never support --user. The user install scheme predates virtual environments. If virtual environments existed today, but the user install scheme didn't, I'd be very surprised if the user install scheme would be added to the packaging ecosystem.

[matthew-brett]

To clarify, there's a good chance that we'll never support --user. The user install scheme predates virtual environments. If virtual environments existed today, but the user install scheme didn't, I'd be very surprised if the user install scheme would be added to the packaging ecosystem.

That's quite possible - requiring virtualenvs was clearly attractive to the pip maintainers - as seen here : https://peps.python.org/pep-0704 . I suspect the reason that --user installs have persisted is the one I and several others have given above - virtualenvs are hard to explain to beginners, and a frequent source of confusion - but that is much less true for adding --user to pip install.

[Malix-Labs]

Requiring venv by default but allowing user install with an explicit --user flag seems to be the best of both worlds

Again, venv is bloat in devcontainers

[matthew-brett]

Requiring venv by default but allowing user install with an explicit --user flag seems to be the best of both worlds

Requiring venvs by default means that the users who are most likely to be confused will get the default behavior, and have to work out how to avoid it. --user is better than nothing for those users, but having a default - invisible to user - virtualenv is even better, in my opinion.

[Malix-Labs]

Requiring venvs by default means that the users who are most likely to be confused will get the default behavior, and have to work out how to avoid it.

having a default - invisible to user - virtualenv is even better, in my opinion.

So you want

• uv pip install to execute uv venv && source .venv/bin/activate if no venv is already present
• uv pip install --user to allow install in user's python

?

[matthew-brett]

So you want

* `uv pip install` to execute `uv venv && source .venv/bin/activate` if no venv is already present

* `uv pip install --user` to allow install in user's python

No - not uv venv && source .venv/bin/activate - because part of the problem with virtualenvs is explaining to the user why they have to do .venv/bin/activate to get their installed packages. With that result, when the beginner closes their terminal and start it again, perhaps with another working directory, they'll be in some trouble.

What I would like is for uv pip install pygame to just work(TM) for a beginner user, with no further setup, such that, when they close their terminal and start it again later, they can do e.g. import pygame.

One way that might work, is to follow the conda pattern - there is a default environment, that the user is installing into, often without them realizing it. When they get more advanced they can make new environments and switch to them.

For pip, I am wondering if this can be achieved with a default virtualenv, say in ~/.venvs/default. When the beginner does uv pip install pygame, instead of seeing error: No Python interpreters found in virtual environments, uv creates ~/.venvs/default if necessary, installs into that virtualenv, and activates it (as you've suggested, except with the standard location). There is some default or simply-enabled setup that activates this virtualenv by default on starting a terminal, if no other is activated - so the user will by default be able to restart the terminal and do python -c "import pygame".

With that working, I think --user becomes an unnecessary confusion, and I'd be happy to see it go.

[potiuk]

Actually as an author of the issue I stopped needing it. And @charliermarsh @zanieb - feel free to close that one if you feel that you do not want to implement --user flag.

I was in the same camp "venv in container images is a bloat" - and this was for me the main reason why I opened the issue, but since then I found out that when I create venv in ~./.local - and simply set VIRTUAL_ENV vatiable to point to it and having ~/.local/bin on the PATH as first entry does precisely what everyone needs here:

• no need to explain user to do activate
• activate merely sets the VIRTUAL_ENV variable and inserts PATH entry - and this is as specified in venv defintion, so you can do precisely the same in your container image or environment definition
• it also nicely retains packages installed in .local when you create a new virtual environment with --system flag (which was the reason why I opened the issue) - which means that you can nicely create and activate new venvs from such .local venv
• this all adds virtually no overhead comparing to --user flag

I believe now that there is no particular reason to have --user flag, really. People who say that they do not want to maintain extra scripts are really talking about running two commands. People mentioned in the discussion thread here, that they do not want to teach their users about venv, and the approach above 100% solves that. You don't have to. You just create the venv and set the right environments for them and they can continue using uv pip install or pip install and it will just work. Mission accomplished.

[potiuk]

BTW. Kind request - If someone would like to discuss whether to use some .venv by default, this is NOT the right place to discuss it. This is a discussion about the --user flag. Whether to use a venv by default is very tangential to the --user flag and I think if someone would like to discuss it, they should open their own issue/discussion, rather than hijack this issue - which was initially on ly for --user support.

[Malix-Labs]

The associated pull request was just closed minutes ago

Does that mean that devcontainers and codespaces will never be able to use uv without the venv bloat?

[zanieb]

We didn't close the pull request, the author did because they have a clear work around.

I think the trade-offs discussed here are not suggesting that we should add this flag. I don't think that means that we'll definitely never add the flag, just that the current evidence is not in favor of it.

I don't really see "virtual environment bloat" as a real problem, they're very lightweight and if you want to bypass it there are a couple workarounds. If this is a big deal to you, I think you'll need to provide more information on why.

What I would like is for uv pip install pygame to just work(TM) for a beginner user, with no further setup, such that, when they close their terminal and start it again later, they can do e.g. import pygame

This is the most compelling argument for user-level installations, but mutable global environments have been repeatedly shown to be the source of a great deal of confusion and problems. We'd like to solve this problem in other ways.

[potiuk]

Does that mean that devcontainers and codespaces will never be able to use uv without the venv bloat?

My experience is tha venv is about the same bloat as what gets created with --user flag. But this has been discussed here and elsewhere - i.e. approach where venv will be created automatically for you if not existing. That seems much more consistent approach.

Though I think it's the question or where and how it should be created. I think it's quite appealting and constructive discussing that possibility is probably best we all can do.

As the author of that issue, I am also OK with closing this one, because I also agree with @zanieb that just "straightforward" --user flag rip-off from pip is not the best appraoch.

Ideally, maybe we should discuss and agree a new PEP how venvs should be automatically created for the out-of-the-box cases?

[matthew-brett]

Just to answer the previous post - why mix the discussion of --user and default virtualenvs? - because, if there are no immediate plans for a "just-works" solution for the beginner, such as default virtualenv, then I would be interested in having the --user option as a not-quite-as-good solution to the same problem - even if it was only temporary.