pip install inside nox-poetry test complains about missing distutils.cmd

[arwedus]

Issue

With this noxfile.py:

import nox  # noqa: F401
from nox_poetry import session

# The versions here must be in sync with the versions in the Dockerfile
PYTHON_VERSIONS = ["3.8", "3.10", "3.11"]
SPHINX_NEEDS_VERSIONS = ["1.3.0", "2.0.0"]

@session(python=PYTHON_VERSIONS)
@nox.parametrize("sphinx_needs", SPHINX_NEEDS_VERSIONS)
def tests(session, sphinx_needs):
    """Run the test suite."""
    session.install(".[testing]", ".")
    session.install(f"sphinx-needs~={sphinx_needs}")
    posargs = session.posargs or ["tests"]
    session.run("pytest", "-v", *posargs, external=True)

I get these errors when running `nox:

nox > Command python -m pip install --constraint=.nox/tests-3-8-sphinx_needs-2-0-0/tmp/requirements.txt distutils failed with exit code 1:
Traceback (most recent call last):
  File "/usr/lib/python3.8/runpy.py", line 194, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/usr/lib/python3.8/runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "/workspaces/yaaa2sphinx-needs/.nox/tests-3-8-sphinx_needs-2-0-0/lib/python3.8/site-packages/pip/__main__.py", line 22, in <module>
    from pip._internal.cli.main import main as _main
  File "/workspaces/yaaa2sphinx-needs/.nox/tests-3-8-sphinx_needs-2-0-0/lib/python3.8/site-packages/pip/_internal/cli/main.py", line 10, in <module>
    from pip._internal.cli.autocompletion import autocomplete
  File "/workspaces/yaaa2sphinx-needs/.nox/tests-3-8-sphinx_needs-2-0-0/lib/python3.8/site-packages/pip/_internal/cli/autocompletion.py", line 10, in <module>
    from pip._internal.cli.main_parser import create_main_parser
  File "/workspaces/yaaa2sphinx-needs/.nox/tests-3-8-sphinx_needs-2-0-0/lib/python3.8/site-packages/pip/_internal/cli/main_parser.py", line 9, in <module>
    from pip._internal.build_env import get_runnable_pip
  File "/workspaces/yaaa2sphinx-needs/.nox/tests-3-8-sphinx_needs-2-0-0/lib/python3.8/site-packages/pip/_internal/build_env.py", line 19, in <module>
    from pip._internal.cli.spinners import open_spinner
  File "/workspaces/yaaa2sphinx-needs/.nox/tests-3-8-sphinx_needs-2-0-0/lib/python3.8/site-packages/pip/_internal/cli/spinners.py", line 9, in <module>
    from pip._internal.utils.logging import get_indentation
  File "/workspaces/yaaa2sphinx-needs/.nox/tests-3-8-sphinx_needs-2-0-0/lib/python3.8/site-packages/pip/_internal/utils/logging.py", line 29, in <module>
    from pip._internal.utils.misc import ensure_dir
  File "/workspaces/yaaa2sphinx-needs/.nox/tests-3-8-sphinx_needs-2-0-0/lib/python3.8/site-packages/pip/_internal/utils/misc.py", line 44, in <module>
    from pip._internal.locations import get_major_minor_version
  File "/workspaces/yaaa2sphinx-needs/.nox/tests-3-8-sphinx_needs-2-0-0/lib/python3.8/site-packages/pip/_internal/locations/__init__.py", line 66, in <module>
    from . import _distutils
  File "/workspaces/yaaa2sphinx-needs/.nox/tests-3-8-sphinx_needs-2-0-0/lib/python3.8/site-packages/pip/_internal/locations/_distutils.py", line 20, in <module>
    from distutils.cmd import Command as DistutilsCommand
ModuleNotFoundError: No module named 'distutils.cmd'

It only happens in Python 3.8.

I am running with poetry inside a Ubuntu 22.04 based docker container that contains the python versions 3.10 (system default), 3.8, and 3.11 (both installed via deadsnakes as apt packages). I have also installed the pseudo package python-is-python3 to have "python" symlink available in /usr/bin.

When I try sudo apt install python3-distutils in the devcontainer, apt just tells me it's already installed. So it should be available to pip running inside nox-poetry too.

What's going on here?

p.s.: Here's an excerpt of my pyproject.toml:

[tool.poetry.dependencies]
python = ">=3.8,<3.13"
PyYAML = "^5.4.1"
sphinx-needs = ">=1.0"

[tool.poetry.group.testing.dependencies]
# vscode testing extensions requires pytest modules to be loadable in default virtualenv (.venv). Therefore
# we unfortunately cannot install it in the devcontainer via pipx.
pytest = "*"
pytest-cov = "*"
pytest-emoji = "*"
pytest-md = "*"
``

[cjolowicz]

Have you tried apt install python3.8-distutils?

[arwedus]

@cjolowicz : I've done that in the Dockerfile now (also from deadsnakes), and it solved the issue. So it was actually totally unrelated to nox-poetry. Thanks for your super fast answer, though!

So, what was actually the root cause is distutils has to be installed in a separate package for every python version in Ubuntu Linux (from my web search, I've found that the Python maintainers are not really happy with this debian packaging decision, and I can see why). For the system python version, the python3-distutils package was already pre-installed.

[cjolowicz]

Glad this solved the issue. For Python development on Debian, use python3-full, it also depends on python3-distutils. Deadsnakes works the same way.