pip user config is invisible when PIP_CONFIG_FILE is set

ccoulombe commented 1 week ago

Description

The user configuration settings are ignored when PIP_CONFIG_FILE is set to an existing config file.

~ ➜ PIP_CONFIG_FILE=~/Projects/test/test-no-user/pip.conf pip config debug 
env_var:
  PIP_CONFIG_FILE='/home/charles/Projects/test/test-no-user/pip.conf'
env:
  /home/charles/Projects/test/test-no-user/pip.conf, exists: True
    install.only-binary: numpy,scipy,mpi4py,pandas,h5py,Cython,grpcio
    install.prefer-binary: true
    install.disable-pip-version-check: true
    install.require-virtualenv: true
global:
  /etc/xdg/pip/pip.conf, exists: False
  /etc/pip.conf, exists: False
site:
  /usr/pip.conf, exists: False

and when PIP_CONFIG_FILE is not set:

~ ➜ pip config debug     
env_var:
env:
global:
  /etc/xdg/pip/pip.conf, exists: False
  /etc/pip.conf, exists: False
site:
  /usr/pip.conf, exists: False
user:
  /home/charles/.pip/pip.conf, exists: False
  /home/charles/.config/pip/pip.conf, exists: False

In other words, having a per-user configuration file and PIP_CONFIG_FILE together is not possible.

Expected behavior

Able to have different configuration file (global, site, per-user and defined by PIP_CONFIG_FILE) so they can be combined as described in the documentation (if I understand correctly).

When multiple configuration files are found, pip combines them in the following order: Global User Site PIP_CONFIG_FILE, if given.

pip version

23.2.1 - 24.2

Python version

3.10 - 3.12

OS

Linux

How to Reproduce

Symptom 1

unset PIP_CONFIG_FILE variable
create a user config file under $HOME/.config/pip/pip.conf
run pip config debug and see the user config being listed
set PIP_CONFIG_FILE to a valid config file : export PIP_CONFIG_FILE=~/pip.conf
run pip config debug and see the user config not listed

pip config used:

➜ cat pip.conf 
[install]
only-binary = numpy,scipy,mpi4py,pandas,h5py,Cython,grpcio
prefer-binary = true
disable-pip-version-check = true
require-virtualenv = true

Symptom 2

One can reproduce the same issue as well by getting a parameter :

set PIP_CONFIG_FILE to a valid configuration file
get a config parameter, eg pip config get install.constraint (returns an error)
set PIP_CONFIG_FILE to empty string
get a config parameter, eg pip config get install.constraint (returns the value)

$ cat ~/.pip/pip.conf 
[install]
constraint = $HOME/.config/pip/fonttools-constraints.txt
require-virtualenv = true

Output

Symptom 1

test-no-user ➜ pip config debug
env_var:
env:
global:
  /etc/xdg/pip/pip.conf, exists: False
  /etc/pip.conf, exists: False
site:
  /usr/pip.conf, exists: False
user:
  /home/charles/.pip/pip.conf, exists: False
  /home/charles/.config/pip/pip.conf, exists: False
test-no-user ➜ PIP_CONFIG_FILE=pip.conf pip config debug 
env_var:
  PIP_CONFIG_FILE='pip.conf'
env:
  pip.conf, exists: True
    install.only-binary: numpy,scipy,mpi4py,pandas,h5py,Cython,grpcio
    install.prefer-binary: true
    install.disable-pip-version-check: true
    install.require-virtualenv: true
global:
  /etc/xdg/pip/pip.conf, exists: False
  /etc/pip.conf, exists: False
site:
  /usr/pip.conf, exists: False

Symptom 2

~ $ cat ~/.pip/pip.conf 
[install]
constraint = $HOME/.config/pip/fonttools-constraints.txt
require-virtualenv = true

~ $ pip config get install.constraint
ERROR: No such key - install.constraint
~  $ PIP_CONFIG_FILE= pip config get install.constraint
$HOME/.config/pip/fonttools-constraints.txt

Code of Conduct

[X] I agree to follow the PSF Code of Conduct.

ccoulombe commented 1 week ago

I've found this comment from https://github.com/pypa/pip/pull/11850

per-user config is not loaded when env_config_file exists

but without a clear explanation as to why that is ? The documentation does not mention a reason either, but is clear that they are combined starting from global > user > site > pip_config_file env variable and overriding existing settings along the way.

Maybe @dalebrydon would have some ideas/insights on this issue?

dalebrydon commented 1 week ago

I will mention that this is currently documented:

Note that if a file exists at the location that [PIP_CONFIG_FILE] is set to, the user config file will not be loaded.

Here's what I imagine the thinking that went into this is: PIP_CONFIG_FILE is meant as a way to specify an alternate path to the user config file and make it the most important config. The behaviour of this can probably not be changed now. One could potentially add another environment variable that would have your desired functionality.

Note that site config has a confusing name and is really meant to be project-level config. If you create a virtual environment and stick everything you want in a pip config there, then you'd get the desired functionality (user config still loaded but overridden by another config).

pypa / pip