pylint-dev / pylint

It's not just a linter that annoys you!
https://pylint.readthedocs.io/en/latest/
GNU General Public License v2.0
5.33k stars 1.14k forks source link

Calculate linter.config.jobs in cgroupsv2 environments #10089

Open DominicLavery opened 4 days ago

DominicLavery commented 4 days ago

Type of Changes

Type
:bug: Bug fix
:sparkles: New feature

Description

In containers running on cgroupv2 systems _query_cpu currently returns None. This results in sched_getaffinity being used, which will normally return all installed CPUs of the host. This can result in crashes with the error:

concurrent.futures.process.BrokenProcessPool: A process in the process pool was terminated abruptly while the future was running or pending.

The changes here use the CPU quota in v2 systems. max 100000 is the default value, and will continue to result in the hosts CPU count being used.

cpu.weight (the replacement of cpu shares from v1) could also be used, but as it's impact on CPU scheduling is relative to the rest of the cgroup hierarchy it isn't possible to get an accurate value on all systems where pylint may be run. Whereas this method is reliable on any container with a CPU quota

github-actions[bot] commented 4 days ago

🤖 According to the primer, this change has no effect on the checked open source code. 🤖🎉

This comment was generated for commit ff64dd84af8ee9afd0a345d248054bddfc73b0b4

codecov[bot] commented 2 days ago

Codecov Report

All modified and coverable lines are covered by tests :white_check_mark:

Project coverage is 95.80%. Comparing base (68cb5b3) to head (ab7b61b).

Additional details and impacted files [![Impacted file tree graph](https://app.codecov.io/gh/pylint-dev/pylint/pull/10089/graphs/tree.svg?width=650&height=150&src=pr&token=ZETEzayrfk&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=pylint-dev)](https://app.codecov.io/gh/pylint-dev/pylint/pull/10089?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=pylint-dev) ```diff @@ Coverage Diff @@ ## main #10089 +/- ## ======================================= Coverage 95.80% 95.80% ======================================= Files 174 174 Lines 18962 18972 +10 ======================================= + Hits 18166 18176 +10 Misses 796 796 ``` | [Files with missing lines](https://app.codecov.io/gh/pylint-dev/pylint/pull/10089?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=pylint-dev) | Coverage Δ | | |---|---|---| | [pylint/lint/run.py](https://app.codecov.io/gh/pylint-dev/pylint/pull/10089?src=pr&el=tree&filepath=pylint%2Flint%2Frun.py&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=pylint-dev#diff-cHlsaW50L2xpbnQvcnVuLnB5) | `88.48% <100.00%> (+0.89%)` | :arrow_up: |

🚨 Try these New Features:

github-actions[bot] commented 2 days ago

🤖 According to the primer, this change has no effect on the checked open source code. 🤖🎉

This comment was generated for commit ab7b61b0ea4134c577bc221ec1eae80c98210459