Bug: console_scripts installed even when extras not requested

[Kixunil]

Summary

When [some_extra] is present in console_scripts and the package is installed using pip3 install package (without [some_extra]), the script is still installed. If the extra requires additional dependencies, then the resulting command is broken.

Steps to reproduce

Steps using a real-life example

0. Get a system with git and pip3 installed, but without click, requests or toml
1. git clone -b cli https://github.com/Kixunil/lnurl
2. cd lnurl
3. sudo pip3 install lnurl
4. lnurl

Expected behavior:

lnurl: command not found error, no lnurl in /usr/local/bin

What happens

lnurl command actually exists and runs throwing exceptions about missing dependencies. Of course sudo pip3 install .[cli] installs the required dependencies, so lnurl runs fine - this works as intended.

Meta

Python version: 3.7 OS: Debian 10

[jordiandreu]

According to setuptools documentation:

Extras can be used by a project’s entry points to specify dynamic dependencies. For example, if Project A includes a “rst2pdf” script, it might declare it like this, so that the “PDF” requirements are only resolved if the “rst2pdf” script is run:

setup(
    name="Project-A",
    ...
    entry_points={
        "console_scripts": [
            "rst2pdf = project_a.tools.pdfgen [PDF]",
            "rst2html = project_a.tools.htmlgen",
            # more script entry points ...
        ],
    }
)

I expect the command rst2pdf to be created when Project-A is installed. However, if it is not, then one could install the dependencies defined by [PDF] later on but the command would not exist and you will need to reinstall Project-A again to get the command rst2pdf.

Nevertheless, IMHO what I think it deserves an clarification is the sentence:

the “PDF” requirements are only resolved if the “rst2pdf” script is run

What does resolved means here? I would expect setuptools to take care about the missing dependencies when running the command and report an runtime error about how to install the required dependencies defined in Project-A, under the key [PDF].

Currently, one can run the command normally and it will fail with a ModuleNotFound exception as soon as on tries to import any of the modules included as a dependency in [PDF].

This is the case in GEPace project (Thanks @tiagocoutinho)

pip install gepace GEPace

Traceback (most recent call last): File "/local/miniconda3/envs/test_setup_37/bin/GEPace", line 5, in from gepace.tango.server import main File "/local/miniconda3/envs/test_setup_37/lib/python3.7/site-packages/gepace/tango/server/init.py", line 1, in from .pace import Pace File "/local/miniconda3/envs/test_setup_37/lib/python3.7/site-packages/gepace/tango/server/pace.py", line 3, in import tango ModuleNotFoundError: No module named 'tango'

The key tango lists pytango (tango) as a dependency, which is not managed by setuptools and an exception is raised during runtime.

[rupertnash]

I came on here to report this issue, but re-reading the docs, I see it's more complex.

The documentation sayeth: https://setuptools.readthedocs.io/en/latest/userguide/entry_point.html#dependency-management

Some entry points may require additional dependencies to properly function. For such an entry point, declare in square brakets any number of dependency extras following the entry point definition. Such entry points will only be viable if their extras were declared and installed. See the guide on dependencies management for more information on defining extra requirements. Consider from the above example:

[options.entry_points]
console_scripts =
hello-world = timmins:hello_world [pretty-printer]

In this case, the hello-world script is only viable if the pretty-printer extra is indicated, and so a plugin host might exclude that entry point (i.e. not install a console script) if the relevant extra dependencies are not installed.

(emphasis mine)

So what does "viable" mean here? "might exclude" is also unhelpful language.

It's clear that the docs need to be clarified, but to what? This is reasonably complex because setuptools is not pip (/poetry/flit/etc). Does setup tools want to mandate what installers do here? Should they skip the script? (My preferred option - I see the argument above but disagree) Should they write a script that checks for the required packages (maybe with importlib.metadata)? The current state (with pip) of writing a script that fails with ModuleNotFoundError just makes the package look like its setup is broken.

PS: spotted and fixed a typo (#2779) while replying here :)

[Kixunil]

you will need to reinstall Project-A again

This feels more reasonable to me than the current situation. If I requested specifically to not install it then I don't want it. Compare it to other dependency managements such as apt - you don't have random non-functioning binaries installed to be prepared for the possibility of installing a library that's used by them later.

might exclude that entry point (i.e. not install a console script)

Hmm, missed that one. So it's correct per that documentation but as I explained above it'd be preferable to exclude the console script.