pip install not working while installing packages defined in setup.py

[Niladri24dutta]

Description

This issue is happening since today morning and our CI pipelines are failing because of this . We are trying to install python packages which are defined in our setup.py file using the below command -

python -m pip install wheel setuptools --index-url <some Url>
python -m pip install '.[testing_req]' --index-url <some Url>

"testing_req" is defined under extras_require argument of setup() . Here is how the setup.py file looks like -

 setup(
  name="some package",
    ...,
    install_requires=[
       list of python package to be installed
    ],
  extras_require={
        "testing_req": [list of python package to be installed for testing],
    }
)

I have observed that if I use the above pip install command it is only installing the packages which are present under "install_requires" but it is ignoring all the packages which are mentioned under "extras_require" . It should also install the additional packages apart from the required packages. Currently as a workaround I had to provide the path to the requirements.txt file to install the packages.

I just noticed that there was a new version released for "wheel" yesterday which the below changes -

 Canonicalized requirements in METADATA file
  Deprecated the bdist_wheel module, as the code was migrated to setuptools itself

Here is the release https://github.com/pypa/wheel/releases/tag/0.44.0 . Could there be any conflict due to this?

Please find the versions used below

• Python version 3.9.19
• pip version 21.3.1
• wheel version 0.44.0/ 0.43.0 (tested with both same issue)
• setuptools version 58.1.0

Expected behavior

While using the below command -

python -m pip install '.[testing_req]' --index-url <some Url>

It should also install the additional packages apart from the required packages. But it is only installing the packages which are present under "install_requires" but it is ignoring all the packages which are mentioned under "extras_require"

pip version

21.3.1

Python version

3.9.19

OS

Linux(Ubuntu 22.04)

How to Reproduce

1. Create setup.py file using the above structure mentioned.
2. Run the below commands to install wheel,setuptools and the python packages -

   python -m pip install wheel setuptools --index-url <some Url>
   python -m pip install '.[testing_req]' --index-url <some Url>

Output

Observe that additional packages mentioned under "extras_require" are not installed . Only the packages under "install_requires" are getting installed.

Code of Conduct

• [X] I agree to follow the PSF Code of Conduct.

[notatallshaw]

pip version

21.3.1

Do you see this same issue with the latest version of pip? (24.2)

[Niladri24dutta]

Do you see this same issue with the latest version of pip? (24.2)

@notatallshaw Unfortunately upgrading the pip to 24.2 is not an option for me , pip is not allowing wildcard() in the package version dependency in the latest version. We are installing a specific version of a package which has wildcard() in one of its dependency version. So we are not able to install the package.

[Niladri24dutta]

@notatallshaw I have upgraded and tested with setuptools 72.1.0 as mentioned here . https://github.com/pypa/setuptools/issues/4543 .Still the same issue.

[Niladri24dutta]

@notatallshaw I have figured out a work around for this . I am not sure if this is expected way of installing the extra packages or not in the new version. This could also be due to the latest version of wheel (0.44.0). Here is what worked for me - Instead of using this command python -m pip install '.[testing_req]' --index-url <some Url> It works if I use this command python -m pip install '<my package name in setup.py>[testing_req]' --index-url <some Url>

I am still not sure about the root cause

[uranusjr]

A reproducer is needed as mentioned here https://github.com/pypa/setuptools/issues/4543#issuecomment-2270767650

My intuition is this is an extra normalisation issue. Likely the built wheel does not specify the extra name correctly? I’m not going into details until a reproducer is available though.