Open cburroughs opened 2 weeks ago
Just a thought on this...
It's not a bad idea in general, but I'm always hesitant to try to outsmart the OS's scheduler. Any OS. In the past when I've done this myself, or used tooling that did, inevitably something in the OS changes and we're now stuck with a maintenance burden - or we've become a detriment someone's workflow.
There is probably some prior art here, in Dockerland maybe, or other projects which are multi-core and CPU intensive.
There might also be the future burden of over-customization requests (e.g. I want these cores for this goal only under these conditions). We have this to a small degree today, and if we start pinning cores 🤷🏽
Is your feature request related to a problem? Please describe.
Circa 2024 a not uncommon workstation is an Apple M3 with a mix of "performance" and "efficiency" cores. (Some
arm64
andx86_64
cpus also have a similar split.)The various options that have cpu core based heuristics (ex:
process_execution_local_parallelism
) would may be better served by being based only on performanceDescribe the solution you'd like Pants auto-magically makes great choices.
Describe alternatives you've considered Encouraging manual setting of the above settings in cases where the number of "total cores" seems high relative to DRAM.
Additional context See also #15840 for another reasonable heuristic adjustment.