installing hook on pre-commit fails with newest virtualenv (>20.24.5) present

fizyk commented 8 months ago

virtualenv 20.24.6 updates embed setuptools from 6.2.0 to 6.2.2 and pip to 23.3.1

pip 23.3 includes some changes to dependency resolution, so that's what might cause the issue.

Pre-commit config:

  - repo: https://github.com/jorisroovers/gitlint
    rev: v0.19.1
    hooks:
      - id: gitlint
      - id: gitlint-ci
        stages:
          - manual
        args: [--commits, origin/master..HEAD]

results of running on a clean pre-commit env (cleared pre-commit cache)

❯ pre-commit run --hook-stage manual gitlint-ci                                                                                                                                                                                    ─╯
[INFO] Installing environment for https://github.com/jorisroovers/gitlint.
[INFO] Once installed this environment will be reused.
[INFO] This may take a few minutes...
An unexpected error has occurred: CalledProcessError: command: ('[HOME]/.cache/pre-commit/repos8g02w1x/py_env-python3.10/bin/python', '-mpip', 'install', '.', './gitlint-core[trusted-deps]')
return code: 1
stdout:
    Processing /home/fizyk/.cache/pre-commit/repos8g02w1x
      Installing build dependencies: started
      Installing build dependencies: finished with status 'done'
      Getting requirements to build wheel: started
      Getting requirements to build wheel: finished with status 'done'
      Preparing metadata (pyproject.toml): started
      Preparing metadata (pyproject.toml): finished with status 'done'
    Processing ./gitlint-core
      Installing build dependencies: started
      Installing build dependencies: finished with status 'done'
      Getting requirements to build wheel: started
      Getting requirements to build wheel: finished with status 'done'
      Preparing metadata (pyproject.toml): started
      Preparing metadata (pyproject.toml): finished with status 'done'
    INFO: pip is looking at multiple versions of gitlint to determine which version is compatible with other requirements. This could take a while.
stderr:
    ERROR: Could not find a version that satisfies the requirement gitlint-core==0.1.dev1+gacc9d9d (from gitlint) (from versions: 0.17.0, 0.18.0, 0.19.0.dev73, 0.19.0.dev75, 0.19.0.dev76, 0.19.0.dev77, 0.19.0.dev78, 0.19.0.dev79, 0.19.0.dev80, 0.19.0.dev81, 0.19.0.dev82, 0.19.0rc1, 0.19.0rc2.dev1, 0.19.0rc2, 0.19.0rc3.dev1, 0.19.0, 0.19.1.dev1, 0.19.1.dev2, 0.19.1, 0.19.2.dev1, 0.19.2.dev2, 0.19.2.dev3, 0.19.2.dev4, 0.19.2.dev5, 0.20.0.dev2, 0.20.0.dev3, 0.20.0.dev6, 0.20.0.dev7, 0.20.0.dev8, 0.20.0.dev9, 0.20.0.dev10, 0.20.0.dev11, 0.20.0.dev12, 0.20.0.dev13, 0.20.0.dev14, 0.20.0.dev15, 0.20.0.dev16, 0.20.0.dev17, 0.20.0.dev18, 0.20.0.dev19, 0.20.0.dev20, 0.20.0.dev21, 0.20.0.dev22, 0.20.0.dev23, 0.20.0.dev24, 0.20.0.dev25, 0.20.0.dev27, 0.20.0.dev29, 0.20.0.dev30, 0.20.0.dev31, 0.20.0.dev32, 0.20.0.dev33, 0.20.0.dev34, 0.20.0.dev35, 0.20.0.dev36, 0.20.0.dev37, 0.20.0.dev38, 0.20.0.dev39, 0.20.0.dev40, 0.20.0.dev41, 0.20.0.dev42, 0.20.0.dev43, 0.20.0.dev44, 0.20.0.dev45, 0.20.0.dev48)
    ERROR: No matching distribution found for gitlint-core==0.1.dev1+gacc9d9d
Check the log at [HOME]/.cache/pre-commit/pre-commit.log

KochankovID commented 8 months ago

It looks like there is a problem with new pip version. The obvious fix is: pip install virtualenv==v20.24.5 before running pre-commit

Probably it's somehow related to this new feature: https://github.com/pypa/pip/issues/11924

edspc commented 8 months ago

I got this issue with pre-commit installed via Homebrew.

Command to apply the fix from @KochankovID:

$(brew --prefix)/opt/pre-commit/libexec/bin/python3 -m pip install virtualenv==v20.24.5

SterlingPeet commented 8 months ago

I can not use the virtualenv version pinning solution because I cannot easily ask other developers on my team to apply it. However, ~~it appears that by manually specifying the matching gitlint-core version does provide a workaround that does not force other developers to run extra commands~~; albeit at the cost of revisiting the workaround for every gitlint version update until virtualenv/pip is fixed upstream:

- repo: https://github.com/jorisroovers/gitlint
  rev: v0.19.1
  hooks:
  - id: gitlint
    additional_dependencies: [gitlint-core==v0.9.1]
    stages: [commit-msg]

SterlingPeet commented 8 months ago

Upon further testing, my solution seems to cause a version conflict when actually running the hook install. Embarrassingly I was fooled by running pre-commit run --all-files which did not invoke this check.

fizyk commented 7 months ago

20.24.7 fails as well.

sigmavirus24 commented 7 months ago

Rather than posting here, please test and add details to https://github.com/pypa/pip/issues/12372 so pip developers prioritize user needs

vfazio commented 7 months ago

Should we ask pre-commit to pin virtualenv to <=20.24.5 until virtualenv includes a new version of pip without the regression? Or ask pypa to revert the pip change in virtualenv.

sigmavirus24 commented 7 months ago

I doubt you'll have much luck with either. This needs to be fixed in pip

vfazio commented 6 months ago

I mentioned it in https://github.com/pypa/pip/issues/12372#issuecomment-1857964529, but it should be possible to workaround this issue by telling virtualenv the version of pip to use when seeding the venvs:

VIRTUALENV_PIP=23.3.2 pre-commit install-hooks

Only necessary until virtualenv either updates the pip package on its schedule or provides 23.3.2+ by default.

SterlingPeet commented 6 months ago

My workaround (that actually works this time) was to go to the pre-commit git repo for gitlint on my disk, and download the tags for the repository. This allowed setuptools-scm to resolve the desired tag locally while building the wheel. This would seem to imply that the logic could be fixed in several places to provide a better experience:

In pip itself, presumably by consulting the upstream repo for tag/commit associations
In setuptools-scm by checking the origin repository for tags that might be missing in the local repo
In gitlint by providing a fallback version, for this exact scenario where tag is not properly downloaded into the local repo

Fallback configuration is documented for setuptools-scm in the configuration parameters, and updating it could be rolled into the release tagging process. Or, the fallback could be set to something sane but conservative, like the latest major version.

vfazio commented 4 months ago

New virtualenv tag that bumps pip to 24.0 https://github.com/pypa/virtualenv/releases/tag/20.25.1.

sigmavirus24 commented 4 months ago

@vfazio as in that version of pip fixes this bug?

vfazio commented 4 months ago

@sigmavirus24

pip 23.3.2 technically fixed it, but no version of virtualenv was tagged with that version (though it would have slowly picked up the new pip at some point due to it's internal update policy). virtualenv now specifically embeds 24.0, which includes the fix as well.

vfazio commented 4 months ago

For reference: https://github.com/pypa/pip/commit/69b58102f522f0e10d2b3bd31e7b9ee074abd16d

sigmavirus24 commented 4 months ago

So I'll close this then. The resolution being either

use a version of pip older than 23.3, or,
use a version of pip that's at least 23.3.2

jorisroovers / gitlint

installing hook on pre-commit fails with newest virtualenv (>20.24.5) present #535