Open msharp9 opened 9 months ago
Thanks for the clear issue!
I was just about to raise an issue for this exact use case. We also use codeartifact for a few in house packages but mostly pull public PyPi packages via AWS codeartifact.
Anyway +1 to this. Would love to use uv if this can be supported.
We also set index-url
in our venv by pip config --site set install.index-url "http://path/to/internal/pypi/server"
BTW, for Windows this is %APPDATA%\pip\pip.ini
, see https://pip.pypa.io/en/stable/topics/configuration/#location
Can you perhaps add to the README instructions on how to set the index-url? My company uses a proxy repo and it's not obvoius how to read from it. Do we have to supply --index-url
every time we call python -m pip uv
? That seems a bit cumbersome.
Hi @mcrumiller you can use UV_INDEX_URL
instead. We may want to add all of our environment variables to the README, but they are currently listed in the --help
output e.g.
-i, --index-url <INDEX_URL>
The URL of the Python package index (by default: https://pypi.org/simple)
[env: UV_INDEX_URL=]
The environment variables are nice, but reading the pip config file would be very good for enterprises. It would allow a seamless transition to uv without installing additional configuration on the machines.
We'd like the system config to be considered as well (/etc/xdg/pip/pip.conf
on Ubuntu). Our organisation configure a local mirror for all dev machines & users this way.
The environment variables are nice, but reading the pip config file would be very good for enterprises. It would allow a seamless transition to uv without installing additional configuration on the machines.
Exactly, this is what I was expecting from uv, a drop-in replacement with a major performance boost.
This is still under consideration, but the comment at https://github.com/astral-sh/uv/issues/1789#issuecomment-1978810917 explains our current stance well.
Hi @mcrumiller you can use
UV_INDEX_URL
instead. We may want to add all of our environment variables to the README, but they are currently listed in the--help
output e.g.-i, --index-url <INDEX_URL> The URL of the Python package index (by default: https://pypi.org/simple) [env: UV_INDEX_URL=]
I've been trying environment variables but it raises an error related to Unexpected fragment on URL: contact. I've seen issues that have been closed for this, I'm a little bit puzzled here. Using version 0.1.12 on Windows on private Artifactory repository.
@RichardDally could you please create a new issue if this is not related to pip.conf
support?
Okay, we are officially considering support for this feature. Here is my proposal.
pip
supports discovery of configuration files at various locations
pip.conf
pip.conf
requires opt-in e.g. --pip-config <path>
or UV_PIP_CONFIG=<path>
pip
supports all of the command-line options via the configuration file
no-compile
, etc.
find-links
, index-url
, extra-index-url
native-tls
togglepip
supports separate configuration for each sub-command like install
[global]
declaration or in the top-level[global]
section
[global]
at all if a top-level option exists[global]
or top-level options we will warn it is emptyRequire explicit selection of configuration files
pip
supports discovery of configuration files at various locations
- Do not support automated discovery of
pip.conf
- Do not support global / user / site
- Use of
pip.conf
requires opt-in e.g.--pip-config <path>
orUV_PIP_CONFIG=<path>
Maybe UV can auto discovery pip.conf
in a virtual environment?
UV_INDEX_URL
is actually enough for most use cases. I think many want uv to support pip.conf
because they want uv as a drop-in replacement for pip. Without auto-discovery pip.conf
from a virtual env, this cannot be achieved.
Also, in our project's CI job, there're multiple stage install different dependencies into the same .venv
. For example, the bootstrap script may setup the .venv
and install all the essential dependencies. Then in a later stage, the test script will install all the test related dependencies into the .venv
.
By auto-picking up the pip.conf
in the virtual environment, all the jobs in the following stages don't need to worry about which index it should use, which is very convenient for us.
Maybe UV can auto discovery pip.conf in a virtual environment?
What exactly do this mean? There's a pip.conf
file in the .venv
folder?
What exactly do this mean? There's a
pip.conf
file in the.venv
folder?
Yes. If you activate a .venv
then run pip config --site set install.index-url "http://pypi.internal.com"
, there will be a pip.conf
in the .venv
containing following content:
[install]
index-url = http://pypi.internal.com
Later pip
commands will automatically pick up this config file.
Why not just export an environment variable selecting the configuration file? e.g. UV_PIP_CONFIG=./.venv/pip.conf
?
It's convenient when we want to install some libs into a .venv across multiple sessions.
For example, a developer may set up a local .venv
for development and use it for a relatively long time. Then, one day, they may need to update or install a new lib from an internal Pypi index. Since pip can automatically pick up the pip.conf
file in the .venv
folder, they don't need to remember what the index is or where the pip.conf
is located.
This is not a deal breaker. We already use uv
in our project and are very happy with it. But I think it's a nice-to-have feature.
Thanks for the additional details! I don't think we'd launch this feature with support for that, but I'm not fully opposed. I worry about sometimes supporting inference, but having it work in the most-scoped context makes some sense.
Just as another viewpoint for this conversation, as someone who has helped set up a lot of different users Python environments a lot of users get confused by all the different places pip configuration can go, e.g. environmental variables, multiple user directories, multiple system directories, venv directories etc.
I've encountered many time someone not being able to find where a setting was set, and have a broken environment until someone assisted them.
Do not support automated discovery of pip.conf Do not support global / user / site
Maybe one default location can be supported without causing any confusion? I would pick user config as the one config if I had to choose. I see how not picking any discovery is a simple solution, but perhaps there is a benefit if this choice is done well.
With environment variables the issue is e.g. I use fish shell, but sometimes things are run through bash. Then I have to maintain two configs and export variables there. Even for one shell its an extra step that sometimes could be avoided.
The intent is to introduce our own configuration in the future and we'll support automatic discovery of that. We're just not particularly comfortable reading other tool's configuration files without opt-in.
Since I first created the issue there's been a lot of conversation, this is what I would suggest as I've thought about everyone's different viewpoints.
UV should create it's own configuration file and provide automatic discovery for it. There should then be another tool that converts pip configs to uv configs, e.g pip.conf -> uv.conf. This second tool could provide a simple extensible configuration to add default locations. Collect all pip.conf, and create one combined uv.conf to the auto discovery location.
If UV then added support for plugins, then all a user would have to do is pip install both, say, pip install uv uv-pip-conf
. This would make the user experience feel much more like UV is actually a drop in replacement to pip and pip-tools since no flags or environment variables would have to be set.
This would be similar to how flake8 plugins work, simply pip install flake8 and flake8-import-order as an example, and now flake8 automatically checks your codes imports along with other syntax.
@zanieb Slightly off topic, but would you be willing to share an expected timeline for the config file feature? It’s the one thing preventing me from immediately swapping all of my projects to uv. 😅 Many thanks, love the project.
Hey @d-miketa — we don't have a timeline yet. I think it will be relatively straight-forward to implement but I'm giving time for feedback on my proposal which only has four upvotes compared to the ~50 on the original post.
As stated by @PetterS not having built-in support for /etc/pip.conf
will be detrimental for Enterprise adoption.
In our case we drop a configuration in /etc/pip.conf
using Ansible, we then symlink this file to /etc/xdg/pip/pip.conf
for Ubuntu based systems.
After that any Python package installed using ansible.pip module automatically uses the configuration from /etc/pip.conf
, we don't have to expose ENV's or write extra config files.
That requirement sounds more like you need a persistent configuration file rather than a pip.conf specifically - or am I misunderstanding?
@charliermarsh In enterprise environments /etc/pip.conf
is used to managed several things:
User are just able to do "pip install" without having to configure anything since it's managed by the platform via /etc/pip.conf
.
Any progress on this? It's critical for enterprise environments.
The intent is to introduce our own configuration in the future and we'll support automatic discovery of that. We're just not particularly comfortable reading other tool's configuration files without opt-in
Then uv will stay indefinitely a non-viable product for enterprise solutions.
/etc/pip.conf
or even sometimes pip.conf
files getting cp'd inside of the venv folder during the dependency installation phase is an industry-wide practice/trend, for better or for worse.
Regardless, huge fan of the product and thank you immensely for your time and hard work. Just sad to see this will only stay on personal projects.
I think what I don't understand yet is: if you're going through the effort to migrate from pip to uv, why is it any harder to ask that you use (for sake of argument) uv.conf
rather than pip.conf
? Why is that not possible? Like, if I told you that it was the exact same file, but it had to be named uv.conf
, why would that be a problem?
We may be convinced to change the behavior here (I'm genuinely looking for answers to that question), but FWIW I can personally name a bunch of enterprises that are already using uv, so I'm not worried about it only being viable for personal projects :)
@charliermarsh If uv.conf
becomes a thing, and location is predefined, ex /etc/uv.conf
. It should work enterprise environments.
That’s the thing I’m mostly trying to understand. Is it important the file is literally named pip.conf, that it follows the exact same schema, that it’s auto-discovered in the same way? If so, why? (I’m genuinely open to being convinced, it just adds work and a maintenance burden for us so I want to make sure we have the right motivation.)
Or is it that users just need a way to specify persistent configuration? What if we (eg) supported reading pip.conf but you had to set a UV environment variable for it?
I think what I don't understand yet is: if you're going through the effort to migrate from pip to uv, why is it any harder to ask that you use (for sake of argument) uv.conf rather than pip.conf? Why is that not possible? Like, if I told you that it was the exact same file, but it had to be named uv.conf, why would that be a problem?
Apologies I don't think I expressed myself clearly.
While the actual name of the file doesn't necessarily need to be the same, the feature set need to be mirrored and it would be highly desirable for the discovery mechanism to be mimicked. Mostly having a global predefined location (e.g. /etc/pip.conf) and a project-specific/localized one (e.g. inside of venv in its root) is very important..
From the previous posts it didn't feel like there was functionality-mirroring in mind, but from your clarification it does feel like I'm wrong.
FWIW I can personally name a bunch of enterprises that are already using uv, so I'm not worried about it only being viable for personal projects :)
Would this mean there is another way that UV supports trusted-host, index-url AND extra-index-url (fallback)?
In my enterprise environment, sys admins configure /etc/pip.conf
to work with our PyPI mirror. A sizeable portion of our users probably do not really even understand the mirror or bother with pip.conf
since it "just works". Thinking short-term, it would be nice if uv
picked up pip.conf
so that it was a drop-in replacement for pip
in enterprise environments. It wouldn't require convincing the sys admins to create/maintain an additional config file or to teach users how to do it themselves. Longer-term though, as uv
adoption increases, it probably wouldn't be much of an ask for sys admins to create the file. I think supporting pip.conf
would primarily help less comfortable users get up and running with uv
. More advanced users would have little issue figuring it out, using a hypothetical uv.conf
or environmental variables. With that in mind, less comfortable users probably are sticking with pip
in the short-term anyway. More advanced users are probably the people very excited about uv
and jumping into it.
@charliermarsh In our case (a small data point, but still) we don't care that much how the file is named, as long as uv can resolve deps from non pypi sources and it's possible to define those sources within *.conf.
There can be (and have been where I've worked) automated setup scripts that make sure there is a pip.conf
with appropriate settings. If uv
just works with that file the gradual adoption within the enterprise could be quicker.
It's not a dealbreaker of course since we are talking about developers here.
I think what I don't understand yet is: if you're going through the effort to migrate from pip to uv, why is it any harder to ask that you use (for sake of argument)
uv.conf
rather thanpip.conf
? Why is that not possible? Like, if I told you that it was the exact same file, but it had to be nameduv.conf
, why would that be a problem?We may be convinced to change the behavior here (I'm genuinely looking for answers to that question), but FWIW I can personally name a bunch of enterprises that are already using uv, so I'm not worried about it only being viable for personal projects :)
@charliermarsh Just to comment since I came across this post while playing with uv: We also use AWS Codeartifact, which uses short-lived credentials that need to be refreshed every twelve hours. The AWS CLI offers a single helper command that can obtain the credentials and rewrite the index url in pip.conf
(or the equivalent config file for npm, maven, etc.). Obviously AWS currently offers no such support for uv, so it's on the user to update uv.conf
every twelve hours. Not saying this is a huge deal, just something that could be tedious enough to convince an enterprise user to stick with pip. Loving uv so far though!
I also work in a corporate environment where someone else manages pip.conf
on my behalf and I do not have full control over my users' installations.
My short term plan to be able to replace pip
with uv pip
has been a handmade utility that reads from pip.conf
and sets the corresponding uv
environment variables.
I would be very happy, once uv
supports its own configuration file, if there was a command to create / update said configuration file with values from pip.conf
. Something like
uv import-pip-conf --overwrite --ignore-unsupported --pip-conf-file=~/.pip.conf
(I am making it up but you get the idea hopefully)
I also have the short-lived credentials problem in an corporate environment. My pip.conf
file is updated every 8 hours with new credentials. The tools to update these credentials are owned by other teams at my company. It would be more challenging for me to convince them to adopt uv
before I am able to test it out and demonstrate some improvements with existing workflows.
I was wondering if it would it be possible for uv.conf
to reference specific configurations in pip.conf
? For example what if the uv.conf
file contains the line
index-url = ${/path/to/pip.conf:index-url}
Don't care much about specific syntax (which I took from here) but the intent is to reference values in the pip.conf
file explicitly. This way we could keep updating pip.conf
every 8 hours until I can get broad consensus to move to uv.conf
workflows.
FYI, https://github.com/astral-sh/uv/pull/7851 landed, which now allows you to set global configuration for uv.toml
.
FYI, #7851 landed, which now allows you to set global configuration for
uv.toml
.
Just to make this clear since I'm new to the tool. Is it possible to link the pip.conf using the uv.toml? I tried a few things but they don't seem to work.
@charliermarsh In enterprise environments
/etc/pip.conf
is used to managed several things:
- Index-url
- Search-index
- HTTP/HTTPS proxies
- Custom CA's
User are just able to do "pip install" without having to configure anything since it's managed by the platform via
/etc/pip.conf
.
Quoting myself since I just tried using /etc/uv/uv.toml
and some of these options are not even supported.
uv
keeps pushing some of these configs to ENV variables. These are not practical for enteprise environments.
Currently there's no way to set a cert
file for a proxy/index, and there's no way to set http/https
proxies via pyproject.toml
/ uv.toml
Enteprise environments typically set SquidProxy
or Pypi Mirror
. These services are exposed to users with internal TLS certs signed by the Enterprise CA.
tldr: If the ENV's listed here https://docs.astral.sh/uv/configuration/environment/ were supported via uv.toml
and pyproject.toml
it would solve a lot of the use-cases listed on this thread.
@gaby for this use case you can use native-tls setting https://docs.astral.sh/uv/reference/settings/#native-tls
In this case SSL works automatically if your internal cert is in the system's certificate store (which is usually the case in enterprise setups).
@schlamar Yes, that works. But sometimes the Proxy CA is not installed in the system.
Regardless, you can't set http/https proxy via toml
only via ENV.
Pip allows a user to set configuration settings inside a
pip.conf
, usually stored at~/.config/pip/pip.conf
. For example, setting a default index-url. uv seems to ignore these settings completely.This is a problem since enterprise companies typically have their own PyPI images and don't want users to use the public PyPI for security reasons. For example, users who use AWS codeartifact have this config file updated automatically when they run aws login. https://docs.aws.amazon.com/codeartifact/latest/ug/python-configure-pip.html