pre-commit cache contains tests folder

[shepilov-vladislav]

how did you install flake8?

# See https://pre-commit.com for more information
# See https://pre-commit.com/hooks.html for more hooks
repos:
  - repo: https://github.com/pycqa/flake8
    rev: 7.0.0
    hooks:
      - id: flake8
        types: [python]

unmodified output of flake8 --bug-report

{
  "platform": {
    "python_implementation": "CPython",
    "python_version": "3.11.7",
    "system": "Linux"
  },
  "plugins": [
    {
      "plugin": "mccabe",
      "version": "0.7.0"
    },
    {
      "plugin": "pycodestyle",
      "version": "2.11.1"
    },
    {
      "plugin": "pyflakes",
      "version": "3.2.0"
    }
  ],
  "version": "7.0.0"
}

describe the problem

what I expected to happen

unit tests should not break on CI ...

Gitlab CI Config

stages:
- lint
- test

linters:
  image: python:3.11.7
  stage: lint
  variables:
    PIP_CACHE_DIR: "$CI_PROJECT_DIR/.cache/pip"
    PRE_COMMIT_HOME: ${CI_PROJECT_DIR}/.cache/pre-commit
  before_script:
    - pip install --upgrade pre-commit
  script:
    - pre-commit run --all-files
  cache:
    paths:
      - ${PRE_COMMIT_HOME}

tests:
  image: python:3.11.7
  stage: test
  cache:
    paths:
      - .cache/pip
  before_script:
    - pip install -r requirements.txt
  script:
    - pytest --cov

commands ran

...
=========================== short test summary info ============================
FAILED .cache/pre-commit/repo3wf0f0w3/tests/unit/test_file_processor.py::test_processor_split_line - AttributeError: 'FileProcessor' object has no attribute 'multiline_string'
!!!!!!!!!!!!!!!!!!!!!!!!!! stopping after 1 failures !!!!!!!!!!!!!!!!!!!!!!!!!!!
======================== 1 failed, 205 passed in 6.05s =========================
...

This happens because the flake8 test directory gets into the pre-commit cache, while the CI cache is in ./.cache/pre-commit/... and the pytest runner tries to run this test too. I understand that I can change the location of the cache folder, but still it seems to me that this behavior of pre-commit for flake8 is strange and a better solution would be to exclude the tests directory from pre-commit.

[sigmavirus24]

I believe based on the python pre-commit documentation that if I read between the lines there's a git clone (or an archive tarball downloaded from the latest tag on GitHub) and then a pip install . in that directory. If the former is more/less true, this isn't a Flake8 only problem. Any python pre-commit hook that includes the tests in its repository will have this issue for you based on where you put the directory.

You can also update how you run pytest. For example you can tell pytest to not recurse the .cache directory https://github.com/PyCQA/flake8/blob/5c52d752e64f210369b37654e7b26ce25e2266d6/pytest.ini#L2

You can also just not put the directory in your current repository root in CI. There's many ways to fix this but I don't think this is either a Flake8 bug or a pre-commit bug.

[asottile]

I also strongly discourage running tests in pre-commit as it's far too slow and will frustrated your contributors