No module named 'setuptools' when installing through pre-commit

[CarlosRDomin]

We recently (within the past 5 days max) started having our CI pipelines fail due to pre-commit + isort raising a ModuleNotFoundError: No module named 'setuptools'. When we comment out the isort hook and leave e.g. black, flake8, etc on, the error goes away.

Steps to reproduce (e.g. inside docker run -it python:3.8 bash, also tested with 3.8.5, 3.8.12, 3.9):

# Create an empty folder and initialize a git repo in it, so pre-commit doesn't complain
mkdir test && cd test
git init

# Install pre-commit
pip install pre-commit

# Create a pre-commit config
cat > .pre-commit-config.yaml <<EOF
repos:
  - repo: https://github.com/pycqa/isort
    rev: 5.10.1  # We have tried with several versions, like 5.7.0, 5.5.2, 5.0.0, all with similar results
    hooks:
      - id: isort
EOF

# Run pre-commit (should succeed, there aren't any files!)
pre-commit run -a

Error details (cat /root/.cache/pre-commit/pre-commit.log)

### version information ``` pre-commit version: 2.16.0 sys.version: 3.8.12 (default, Dec 21 2021, 10:45:09) [GCC 10.2.1 20210110] sys.executable: /usr/local/bin/python os.name: posix sys.platform: linux ``` ### error information ``` An unexpected error has occurred: CalledProcessError: command: ('/root/.cache/pre-commit/repor8uvua1d/py_env-python3/bin/python', '-mpip', 'install', '.') return code: 2 expected return code: 0 stdout: Processing /root/.cache/pre-commit/repor8uvua1d Installing build dependencies: started Installing build dependencies: finished with status 'done' Getting requirements to build wheel: started Getting requirements to build wheel: finished with status 'done' stderr: ERROR: Exception: Traceback (most recent call last): File "/root/.cache/pre-commit/repor8uvua1d/py_env-python3/lib/python3.8/site-packages/pip/_internal/cli/base_command.py", line 164, in exc_logging_wrapper status = run_func(*args) File "/root/.cache/pre-commit/repor8uvua1d/py_env-python3/lib/python3.8/site-packages/pip/_internal/cli/req_command.py", line 205, in wrapper return func(self, options, args) File "/root/.cache/pre-commit/repor8uvua1d/py_env-python3/lib/python3.8/site-packages/pip/_internal/commands/install.py", line 338, in run requirement_set = resolver.resolve( File "/root/.cache/pre-commit/repor8uvua1d/py_env-python3/lib/python3.8/site-packages/pip/_internal/resolution/resolvelib/resolver.py", line 73, in resolve collected = self.factory.collect_root_requirements(root_reqs) File "/root/.cache/pre-commit/repor8uvua1d/py_env-python3/lib/python3.8/site-packages/pip/_internal/resolution/resolvelib/factory.py", line 468, in collect_root_requirements req = self._make_requirement_from_install_req( File "/root/.cache/pre-commit/repor8uvua1d/py_env-python3/lib/python3.8/site-packages/pip/_internal/resolution/resolvelib/factory.py", line 430, in _make_requirement_from_install_req cand = self._make_candidate_from_link( File "/root/.cache/pre-commit/repor8uvua1d/py_env-python3/lib/python3.8/site-packages/pip/_internal/resolution/resolvelib/factory.py", line 201, in _make_candidate_from_link self._link_candidate_cache[link] = LinkCandidate( File "/root/.cache/pre-commit/repor8uvua1d/py_env-python3/lib/python3.8/site-packages/pip/_internal/resolution/resolvelib/candidates.py", line 281, in __init__ super().__init__( File "/root/.cache/pre-commit/repor8uvua1d/py_env-python3/lib/python3.8/site-packages/pip/_internal/resolution/resolvelib/candidates.py", line 156, in __init__ self.dist = self._prepare() File "/root/.cache/pre-commit/repor8uvua1d/py_env-python3/lib/python3.8/site-packages/pip/_internal/resolution/resolvelib/candidates.py", line 225, in _prepare dist = self._prepare_distribution() File "/root/.cache/pre-commit/repor8uvua1d/py_env-python3/lib/python3.8/site-packages/pip/_internal/resolution/resolvelib/candidates.py", line 292, in _prepare_distribution return preparer.prepare_linked_requirement(self._ireq, parallel_builds=True) File "/root/.cache/pre-commit/repor8uvua1d/py_env-python3/lib/python3.8/site-packages/pip/_internal/operations/prepare.py", line 482, in prepare_linked_requirement return self._prepare_linked_requirement(req, parallel_builds) File "/root/.cache/pre-commit/repor8uvua1d/py_env-python3/lib/python3.8/site-packages/pip/_internal/operations/prepare.py", line 546, in _prepare_linked_requirement dist = _get_prepared_distribution( File "/root/.cache/pre-commit/repor8uvua1d/py_env-python3/lib/python3.8/site-packages/pip/_internal/operations/prepare.py", line 58, in _get_prepared_distribution abstract_dist.prepare_distribution_metadata(finder, build_isolation) File "/root/.cache/pre-commit/repor8uvua1d/py_env-python3/lib/python3.8/site-packages/pip/_internal/distributions/sdist.py", line 47, in prepare_distribution_metadata self._install_build_reqs(finder) File "/root/.cache/pre-commit/repor8uvua1d/py_env-python3/lib/python3.8/site-packages/pip/_internal/distributions/sdist.py", line 106, in _install_build_reqs build_reqs = self._get_build_requires_wheel() File "/root/.cache/pre-commit/repor8uvua1d/py_env-python3/lib/python3.8/site-packages/pip/_internal/distributions/sdist.py", line 83, in _get_build_requires_wheel return backend.get_requires_for_build_wheel() File "/root/.cache/pre-commit/repor8uvua1d/py_env-python3/lib/python3.8/site-packages/pip/_vendor/pep517/wrappers.py", line 172, in get_requires_for_build_wheel return self._call_hook('get_requires_for_build_wheel', { File "/root/.cache/pre-commit/repor8uvua1d/py_env-python3/lib/python3.8/site-packages/pip/_vendor/pep517/wrappers.py", line 332, in _call_hook raise BackendUnavailable(data.get('traceback', '')) pip._vendor.pep517.wrappers.BackendUnavailable: Traceback (most recent call last): File "/root/.cache/pre-commit/repor8uvua1d/py_env-python3/lib/python3.8/site-packages/pip/_vendor/pep517/in_process/_in_process.py", line 89, in _build_backend obj = import_module(mod_path) File "/usr/local/lib/python3.8/importlib/__init__.py", line 127, in import_module return _bootstrap._gcd_import(name[level:], package, level) File "", line 1014, in _gcd_import File "", line 991, in _find_and_load File "", line 961, in _find_and_load_unlocked File "", line 219, in _call_with_frames_removed File "", line 1014, in _gcd_import File "", line 991, in _find_and_load File "", line 975, in _find_and_load_unlocked File "", line 671, in _load_unlocked File "", line 843, in exec_module File "", line 219, in _call_with_frames_removed File "/tmp/pip-build-env-v50_yrw1/overlay/lib/python3.8/site-packages/poetry/core/masonry/__init__.py", line 10, in from .builder import Builder File "/tmp/pip-build-env-v50_yrw1/overlay/lib/python3.8/site-packages/poetry/core/masonry/builder.py", line 7, in from .builders.sdist import SdistBuilder File "/tmp/pip-build-env-v50_yrw1/overlay/lib/python3.8/site-packages/poetry/core/masonry/builders/__init__.py", line 2, in from .wheel import WheelBuilder File "/tmp/pip-build-env-v50_yrw1/overlay/lib/python3.8/site-packages/poetry/core/masonry/builders/wheel.py", line 23, in from packaging.tags import sys_tags File "/tmp/pip-build-env-v50_yrw1/overlay/lib/python3.8/site-packages/poetry/core/_vendor/packaging/tags.py", line 7, in import distutils.util File "/root/.cache/pre-commit/repor8uvua1d/py_env-python3/lib/python3.8/site-packages/_distutils_hack/__init__.py", line 92, in create_module return importlib.import_module('setuptools._distutils') File "/usr/local/lib/python3.8/importlib/__init__.py", line 127, in import_module return _bootstrap._gcd_import(name[level:], package, level) ModuleNotFoundError: No module named 'setuptools' ``` ``` Traceback (most recent call last): File "/usr/local/lib/python3.8/site-packages/pre_commit/error_handler.py", line 65, in error_handler yield File "/usr/local/lib/python3.8/site-packages/pre_commit/main.py", line 396, in main return run(args.config, store, args) File "/usr/local/lib/python3.8/site-packages/pre_commit/commands/run.py", line 417, in run install_hook_envs(to_install, store) File "/usr/local/lib/python3.8/site-packages/pre_commit/repository.py", line 224, in install_hook_envs _hook_install(hook) File "/usr/local/lib/python3.8/site-packages/pre_commit/repository.py", line 82, in _hook_install lang.install_environment( File "/usr/local/lib/python3.8/site-packages/pre_commit/languages/python.py", line 205, in install_environment helpers.run_setup_cmd(prefix, install_cmd) File "/usr/local/lib/python3.8/site-packages/pre_commit/languages/helpers.py", line 52, in run_setup_cmd cmd_output_b(*cmd, cwd=prefix.prefix_dir, **kwargs) File "/usr/local/lib/python3.8/site-packages/pre_commit/util.py", line 154, in cmd_output_b raise CalledProcessError(returncode, cmd, retcode, stdout_b, stderr_b) pre_commit.util.CalledProcessError: command: ('/root/.cache/pre-commit/repor8uvua1d/py_env-python3/bin/python', '-mpip', 'install', '.') return code: 2 expected return code: 0 stdout: Processing /root/.cache/pre-commit/repor8uvua1d Installing build dependencies: started Installing build dependencies: finished with status 'done' Getting requirements to build wheel: started Getting requirements to build wheel: finished with status 'done' stderr: ERROR: Exception: Traceback (most recent call last): File "/root/.cache/pre-commit/repor8uvua1d/py_env-python3/lib/python3.8/site-packages/pip/_internal/cli/base_command.py", line 164, in exc_logging_wrapper status = run_func(*args) File "/root/.cache/pre-commit/repor8uvua1d/py_env-python3/lib/python3.8/site-packages/pip/_internal/cli/req_command.py", line 205, in wrapper return func(self, options, args) File "/root/.cache/pre-commit/repor8uvua1d/py_env-python3/lib/python3.8/site-packages/pip/_internal/commands/install.py", line 338, in run requirement_set = resolver.resolve( File "/root/.cache/pre-commit/repor8uvua1d/py_env-python3/lib/python3.8/site-packages/pip/_internal/resolution/resolvelib/resolver.py", line 73, in resolve collected = self.factory.collect_root_requirements(root_reqs) File "/root/.cache/pre-commit/repor8uvua1d/py_env-python3/lib/python3.8/site-packages/pip/_internal/resolution/resolvelib/factory.py", line 468, in collect_root_requirements req = self._make_requirement_from_install_req( File "/root/.cache/pre-commit/repor8uvua1d/py_env-python3/lib/python3.8/site-packages/pip/_internal/resolution/resolvelib/factory.py", line 430, in _make_requirement_from_install_req cand = self._make_candidate_from_link( File "/root/.cache/pre-commit/repor8uvua1d/py_env-python3/lib/python3.8/site-packages/pip/_internal/resolution/resolvelib/factory.py", line 201, in _make_candidate_from_link self._link_candidate_cache[link] = LinkCandidate( File "/root/.cache/pre-commit/repor8uvua1d/py_env-python3/lib/python3.8/site-packages/pip/_internal/resolution/resolvelib/candidates.py", line 281, in __init__ super().__init__( File "/root/.cache/pre-commit/repor8uvua1d/py_env-python3/lib/python3.8/site-packages/pip/_internal/resolution/resolvelib/candidates.py", line 156, in __init__ self.dist = self._prepare() File "/root/.cache/pre-commit/repor8uvua1d/py_env-python3/lib/python3.8/site-packages/pip/_internal/resolution/resolvelib/candidates.py", line 225, in _prepare dist = self._prepare_distribution() File "/root/.cache/pre-commit/repor8uvua1d/py_env-python3/lib/python3.8/site-packages/pip/_internal/resolution/resolvelib/candidates.py", line 292, in _prepare_distribution return preparer.prepare_linked_requirement(self._ireq, parallel_builds=True) File "/root/.cache/pre-commit/repor8uvua1d/py_env-python3/lib/python3.8/site-packages/pip/_internal/operations/prepare.py", line 482, in prepare_linked_requirement return self._prepare_linked_requirement(req, parallel_builds) File "/root/.cache/pre-commit/repor8uvua1d/py_env-python3/lib/python3.8/site-packages/pip/_internal/operations/prepare.py", line 546, in _prepare_linked_requirement dist = _get_prepared_distribution( File "/root/.cache/pre-commit/repor8uvua1d/py_env-python3/lib/python3.8/site-packages/pip/_internal/operations/prepare.py", line 58, in _get_prepared_distribution abstract_dist.prepare_distribution_metadata(finder, build_isolation) File "/root/.cache/pre-commit/repor8uvua1d/py_env-python3/lib/python3.8/site-packages/pip/_internal/distributions/sdist.py", line 47, in prepare_distribution_metadata self._install_build_reqs(finder) File "/root/.cache/pre-commit/repor8uvua1d/py_env-python3/lib/python3.8/site-packages/pip/_internal/distributions/sdist.py", line 106, in _install_build_reqs build_reqs = self._get_build_requires_wheel() File "/root/.cache/pre-commit/repor8uvua1d/py_env-python3/lib/python3.8/site-packages/pip/_internal/distributions/sdist.py", line 83, in _get_build_requires_wheel return backend.get_requires_for_build_wheel() File "/root/.cache/pre-commit/repor8uvua1d/py_env-python3/lib/python3.8/site-packages/pip/_vendor/pep517/wrappers.py", line 172, in get_requires_for_build_wheel return self._call_hook('get_requires_for_build_wheel', { File "/root/.cache/pre-commit/repor8uvua1d/py_env-python3/lib/python3.8/site-packages/pip/_vendor/pep517/wrappers.py", line 332, in _call_hook raise BackendUnavailable(data.get('traceback', '')) pip._vendor.pep517.wrappers.BackendUnavailable: Traceback (most recent call last): File "/root/.cache/pre-commit/repor8uvua1d/py_env-python3/lib/python3.8/site-packages/pip/_vendor/pep517/in_process/_in_process.py", line 89, in _build_backend obj = import_module(mod_path) File "/usr/local/lib/python3.8/importlib/__init__.py", line 127, in import_module return _bootstrap._gcd_import(name[level:], package, level) File "", line 1014, in _gcd_import File "", line 991, in _find_and_load File "", line 961, in _find_and_load_unlocked File "", line 219, in _call_with_frames_removed File "", line 1014, in _gcd_import File "", line 991, in _find_and_load File "", line 975, in _find_and_load_unlocked File "", line 671, in _load_unlocked File "", line 843, in exec_module File "", line 219, in _call_with_frames_removed File "/tmp/pip-build-env-v50_yrw1/overlay/lib/python3.8/site-packages/poetry/core/masonry/__init__.py", line 10, in from .builder import Builder File "/tmp/pip-build-env-v50_yrw1/overlay/lib/python3.8/site-packages/poetry/core/masonry/builder.py", line 7, in from .builders.sdist import SdistBuilder File "/tmp/pip-build-env-v50_yrw1/overlay/lib/python3.8/site-packages/poetry/core/masonry/builders/__init__.py", line 2, in from .wheel import WheelBuilder File "/tmp/pip-build-env-v50_yrw1/overlay/lib/python3.8/site-packages/poetry/core/masonry/builders/wheel.py", line 23, in from packaging.tags import sys_tags File "/tmp/pip-build-env-v50_yrw1/overlay/lib/python3.8/site-packages/poetry/core/_vendor/packaging/tags.py", line 7, in import distutils.util File "/root/.cache/pre-commit/repor8uvua1d/py_env-python3/lib/python3.8/site-packages/_distutils_hack/__init__.py", line 92, in create_module return importlib.import_module('setuptools._distutils') File "/usr/local/lib/python3.8/importlib/__init__.py", line 127, in import_module return _bootstrap._gcd_import(name[level:], package, level) ModuleNotFoundError: No module named 'setuptools' ```

[mik-laj]

I also encourage this problem. As workaround, I used the following config:

  - repo: local
    hooks:
      - id: isort
        name: isort
        entry: isort
        require_serial: true
        language: python
        language_version: python3
        types_or: [cython, pyi, python]
        args: ['--filter-files']
        minimum_pre_commit_version: '2.9.2'
        additional_dependencies: ['isort==5.10.1']

[asottile]

the root cause is a bug in pip coupled with some changes in setuptools -- here's proper workarounds: https://github.com/pre-commit/pre-commit/issues/2178#issuecomment-1002163763

[CarlosRDomin]

Thank you for the quick responses everyone! For the readers who get here looking for a solution, I can confirm that all three of the workarounds proposed above work!

That is (either of these will do):

• Using a "local" isort pre-commit hook (thanks @mik-laj for the quick workaround!)
• The MR #1875 by @m-kuhn (tested with pre-commit try-repo), should it get merged? Sounds like a proper long-term solution
• Downgrading virtualenv: pip install 'virtualenv<20.11' (sounds like this will stop working in 28 days, so Jan 21st 2022)

[timothycrosley]

This is not an isort specific issue. isort hasn't made any releases that change this behavior, and in fact hasn't made any release in some months. It also has no external dependencies. isort uses poetry intentionally and not setuptools.This breakage is occurring upstream and I encourage you to reach out to the projects responsible, and in the meantime utilize SETUPTOOLS_USE_DISTUTILS=stdlib or one of the other workarounds @asottile has helpfully cataloged here: pre-commit/pre-commit#2178 (comment)

Will keep this issue open and pinned until it is resolved upstream so that users that run into this issue can quickly find appropriate workarounds.

[m-kuhn]

The underlying issue has been fixed upstream https://github.com/pypa/setuptools/commit/9c9c91c0a3951a7c521bd6eddc3aa5f84f0cfd9f