Add --workers option for multi-process operations - Githubissues

adrienverge / yamllint

A linter for YAML files.

GNU General Public License v3.0

2.85k stars 273 forks source link

Add --workers option for multi-process operations #675

Open dirtyhillbilly opened 4 months ago

dirtyhillbilly commented 4 months ago

This MR allows checking files in a ProcessPoolExecutor With my test base (~6500 files), and quad-core i5 cpu, --workers 0 is twice as fast as sequential operations.

coveralls commented 4 months ago

coverage: 99.476% (-0.3%) from 99.825% when pulling 922347d52c105c2b95d840079faa7bf732960c27 on dirtyhillbilly:feat-processpoolexecutor into 81e9f98ffd059efe8aa9c1b1a42e5cce61b640c6 on adrienverge:master.

coveralls commented 4 months ago

coverage: 99.476% (-0.3%) from 99.825% when pulling 5f8810933f619f654505cadb8ff701317b03b18d on dirtyhillbilly:feat-processpoolexecutor into 81e9f98ffd059efe8aa9c1b1a42e5cce61b640c6 on adrienverge:master.

coveralls commented 4 months ago

coverage: 99.651% (-0.2%) from 99.825% when pulling 9976bff32edf416eb490e0cc8aaf9e90db629249 on dirtyhillbilly:feat-processpoolexecutor into 81e9f98ffd059efe8aa9c1b1a42e5cce61b640c6 on adrienverge:master.

coveralls commented 4 months ago

coverage: 99.677% (-0.1%) from 99.825% when pulling b321c5298854d6b978e0d7e8ca71ba85448a8262 on dirtyhillbilly:feat-processpoolexecutor into 81e9f98ffd059efe8aa9c1b1a42e5cce61b640c6 on adrienverge:master.

coveralls commented 4 months ago

coverage: 99.677% (-0.1%) from 99.825% when pulling e1e0813faa37dab70806ead36208517748f47e47 on dirtyhillbilly:feat-processpoolexecutor into 81e9f98ffd059efe8aa9c1b1a42e5cce61b640c6 on adrienverge:master.

coveralls commented 4 months ago

coverage: 99.677% (-0.1%) from 99.825% when pulling e1e0813faa37dab70806ead36208517748f47e47 on dirtyhillbilly:feat-processpoolexecutor into 81e9f98ffd059efe8aa9c1b1a42e5cce61b640c6 on adrienverge:master.

coveralls commented 4 months ago

coverage: 99.752% (-0.07%) from 99.825% when pulling 41e34b8740d9207d7111c6bbc82aa7058ff54a31 on dirtyhillbilly:feat-processpoolexecutor into 81e9f98ffd059efe8aa9c1b1a42e5cce61b640c6 on adrienverge:master.

coveralls commented 4 months ago

coverage: 99.827% (+0.002%) from 99.825% when pulling 1381c0ba951851cca40fcfd8e9d649ba7493a47c on dirtyhillbilly:feat-processpoolexecutor into 81e9f98ffd059efe8aa9c1b1a42e5cce61b640c6 on adrienverge:master.

coveralls commented 4 months ago

coverage: 99.827% (+0.002%) from 99.825% when pulling 1381c0ba951851cca40fcfd8e9d649ba7493a47c on dirtyhillbilly:feat-processpoolexecutor into 81e9f98ffd059efe8aa9c1b1a42e5cce61b640c6 on adrienverge:master.