autonomys / subspace

Subspace Network reference implementation
https://subspace.network
380 stars 243 forks source link

Improved plotting threads pinning #2963

Closed nazar-pc closed 2 months ago

nazar-pc commented 2 months ago

This improves two specific cases that we might care about.

On Threadripper/Epyc systems (at least, maybe Ryzen too) in contrast to Intel systems logical CPU cores are located after all physical CPU cores. So physical CPU core 0 corresponds to logical CPU cores 0 and 32 on my 7970X instead of cores 0 and 1 that one would expect on Intel system. This meant that simply truncating all CPU cores in half would result in a single logical cores used from each physical CPU core and similar CPU load during replotting to full plotting. This PR changes that to take into account physical CPU cores mapping into logical cores to pick half of physical CPU cores instead as expected.

For hybrid Intel CPUs with P/E cores older code was simply taking first N cores, which always included P-cores, sometimes all P-cores. This resulted in bad user experience when all what was left during replotting was a few E-cores, which resulted in less than optimal experience of using the machine for other tasks. Observation is that both L2 cache size and number of cores per L2 cache size are different on such CPUs, so by grouping CPUs accordingly we can make better decisions. Specifically on systems with 2C4T P-cores and 8C8T E-cores replotting will now use one physical (2 logical) P-cores and 4 E-cores, leaving exactly half of both kinds of cores for other needs.

Space Acres will take advantage of this to make an advanced option available to set number of plotting threads to be the same as number of replotting threads and proper thread pinning is important in that case.

For reference topology on AMD Ryzen Threadripper 7970X:

![7970X](https://github.com/user-attachments/assets/9d578919-61af-4f12-957d-abcf797d720e)

And on Intel Core i7-1360U:

![i7-1360U](https://github.com/user-attachments/assets/9f1a4cbd-b734-4015-b340-9f472b8626a9)

Code contributor checklist:

nazar-pc commented 2 months ago

I tested on my Threadripper CPU and there is no reason it wouldn't work the same exact way on Intel, code is not that complicated. I will ask Jim to test it once integrated into Space Acres either later today or tomorrow.