Pip caching doesn't embed --config-settings arguments, can interfere with subsequent installs (with different args)

[ferdnyc]

Description

As the pygobject-stubs README explains, the package needs to be built with a --config-settings argument specifying which of multiple library versions (Gtk3/Gtk4, Gdk3/Gdk4, Soap2/Soap3, etc.) the types are configured for. This can be done on the pip command line or from a requirements.txt file using --config-settings.

However, as the README also explains, pip install also has to be called with --no-cache-dir on subsequent installs, to prevent it from using a cached build of pygobject-stubs that may not have been configured with the same library selections.

--no-cache-dir can't be specified in a requirements.txt file, and worse it's a global pip option that disables caching for all packages, instead of just the one we want.

(--no-binary appears to be no help here. Binary wheels of pygobject-stubs don't exist, on PyPi, but if pip has created and cached one locally it'll still be pulled from the cache even with --no-binary pygobject-stubs specified.)

Perhaps the cache should be automatically disabled for packages with --config-settings specified, since there's no way to know that a cached build used the same --config-settings as the current package requirement?

Expected behavior

Pip always builds and installs pygobject-stubs with the --config-settings parameters currently specified.

pip version

24.0

Python version

3.12.2

OS

Fedora Linux 39

How to Reproduce

Install two copies of pygobject-stubs, ostensibly configured differently

# Install 1
python3 -m venv venv_first
. ./venv_first/bin/activate
pip install pygobject-stubs --config-settings=config=Gtk3,Gdk3

# Install 2
deactivate
python3 -m venv venv_second
. ./venv_second/bin/activate
pip install pygobject-stubs --config-settings=config=Gtk4,Gdk4

# Verify the stubs versioning...
diff venv_first/lib/python3.12/site-packages/gi-stubs/repository/Gtk.pyi \
  venv_second/lib/python3.12/site-packages/gi-stubs/repository/Gtk.pyi

There should be differences between the two files, because one should have been built with Gtk3 stubs, the other with Gtk4 stubs. However, they will be the same, because the same cached build of the package will have been installed in both venvs.

Replacing both pip install commands with pip install --no-cache-dir, however, will successfully install differently-configured builds of pygobject-stubs.

Output

No response

Code of Conduct

• [X] I agree to follow the PSF Code of Conduct.

[pradyunsg]

Yea, this sounds like a bug and we need to tweak the cache key for such projects to consider the config settings.

[chrysle]

I could have a look.

[sbidoul]

I'm going to merge this in https://github.com/pypa/pip/issues/11164 which has some discussions about various use cases.