pre-commit-ci / issues

public issues for https://pre-commit.ci
17 stars 3 forks source link

pre-commit-ci timeout #187

Closed shaneahmed closed 1 year ago

shaneahmed commented 1 year ago

There is a new issue with running pre-commit on ci introduced during the last two weeks which causes it to fail https://results.pre-commit.ci/run/github/267705904/1694117490.SoArf7wiQSuhKIcMy4vvXw

pre-commit was passing on pre-commit-ci e.g., here https://results.pre-commit.ci/run/github/267705904/1693571232.4atPg5DIQeKIkuwB3M9bSg

but it has started to time out now e.g., https://github.com/TissueImageAnalytics/tiatoolbox/pull/711/files even on small changes like this https://github.com/TissueImageAnalytics/tiatoolbox/pull/712/files

I have run it locally and timed it. You can see the run finishes in less than a minute.

pre-commit run --verbose --all-files
notebook markdown format.................................................Passed
- hook id: notebook-markdown-format
- duration: 1.01s
mdformat.................................................................Passed
- hook id: mdformat
- duration: 0.51s
check python ast.........................................................Passed
- hook id: check-ast
- duration: 0.16s
fix utf-8 byte order marker..............................................Passed
- hook id: fix-byte-order-marker
- duration: 0.12s
check json...........................................(no files to check)Skipped
- hook id: check-json
check for merge conflicts................................................Passed
- hook id: check-merge-conflict
- duration: 0.2s
check for broken symlinks............................(no files to check)Skipped
- hook id: check-symlinks
check toml...............................................................Passed
- hook id: check-toml
- duration: 0.05s
check xml................................................................Passed
- hook id: check-xml
- duration: 0.11s
check yaml...............................................................Passed
- hook id: check-yaml
- duration: 0.1s
detect destroyed symlinks................................................Passed
- hook id: destroyed-symlinks
- duration: 0.23s
detect private key.......................................................Passed
- hook id: detect-private-key
- duration: 0.28s
fix end of files.........................................................Passed
- hook id: end-of-file-fixer
- duration: 0.14s
mixed line ending........................................................Passed
- hook id: mixed-line-ending
- duration: 0.17s
fix requirements.txt.....................................................Passed
- hook id: requirements-txt-fixer
- duration: 0.05s
trim trailing whitespace.................................................Passed
- hook id: trailing-whitespace
- duration: 0.19s
file contents sorter.....................................................Passed
- hook id: file-contents-sorter
- duration: 0.06s
debug statements (python)................................................Passed
- hook id: debug-statements
- duration: 0.19s
pretty format json...................................(no files to check)Skipped
- hook id: pretty-format-json
rst ``code`` is two backticks............................................Passed
- hook id: rst-backticks
- duration: 0.13s
rst directives end with two colons.......................................Passed
- hook id: rst-directive-colons
- duration: 0.09s
rst ``inline code`` next to normal text..................................Passed
- hook id: rst-inline-touching-normal
- duration: 0.13s
black....................................................................Passed
- hook id: black
- duration: 0.19s

All done! \u2728 \U0001f370 \u2728
126 files left unchanged.

black-jupyter............................................................Passed
- hook id: black-jupyter
- duration: 0.44s

All done! \u2728 \U0001f370 \u2728
141 files left unchanged.

ruff.....................................................................Passed
- hook id: ruff
- duration: 0.2s

Not sure what's happening on ci. Please can you help?

shaneahmed commented 1 year ago

To add, the above run was using pre-commit version 3.3.3 on Windows.

To test it on linux, I have set up a new environment on Ubuntu22.04 (WSL on Windows11), again pre-commit version 3.3.3

The test took much longer than what is in the output of verbose.

The output of verbose shows most of the tests finished in less than 2 sec, however the whole run took about 9 mins.

When I was running the test I could see that verbose was showing much less time than taken to run the test.

time pre-commit run --all-files --verbose
notebook markdown format.................................................Passed
- hook id: notebook-markdown-format
- duration: 1.44s
mdformat.................................................................Passed
- hook id: mdformat
- duration: 0.41s
check python ast.........................................................Passed
- hook id: check-ast
- duration: 0.19s
fix utf-8 byte order marker..............................................Passed
- hook id: fix-byte-order-marker
- duration: 0.28s
check json...........................................(no files to check)Skipped
- hook id: check-json
check for merge conflicts................................................Passed
detect private key.......................................................Passed
- hook id: detect-private-key
- duration: 0.39s
fix end of files.........................................................Passed
- hook id: end-of-file-fixer
- duration: 0.29s
mixed line ending........................................................Passed
- hook id: mixed-line-ending
- duration: 0.37s
fix requirements.txt.....................................................Passed
- hook id: requirements-txt-fixer
- duration: 0.04s
trim trailing whitespace.................................................Passed
- hook id: trailing-whitespace
- duration: 1.56s
file contents sorter.....................................................Passed
- hook id: file-contents-sorter
- duration: 0.03s
debug statements (python)................................................Passed
- hook id: debug-statements
- duration: 0.22s
pretty format json...................................(no files to check)Skipped
- hook id: pretty-format-json
rst ``code`` is two backticks............................................Passed
- hook id: rst-backticks
- duration: 0.18s
rst directives end with two colons.......................................Passed
- hook id: rst-directive-colons
- duration: 0.18s
rst ``inline code`` next to normal text..................................Passed
- hook id: rst-inline-touching-normal
- duration: 0.29s
black....................................................................Passed
- hook id: black
- duration: 10.16s

All done! ✨ 🍰 ✨
126 files left unchanged.

black-jupyter............................................................Passed
- hook id: black-jupyter
- duration: 4.33s

All done! ✨ 🍰 ✨
141 files left unchanged.

ruff.....................................................................Passed
- hook id: ruff
- duration: 2.31s

real    9m35.801s
user    0m34.993s
sys     1m7.488s
asottile commented 1 year ago

I get what CI gets locally. if you add exclude_types: [jupyter] to pretty-format-json it is much much faster