search

intel / intel-cmt-cat

User space software for Intel(R) Resource Director Technology
http://www.intel.com/content/www/us/en/architecture-and-technology/resource-director-technology.html
Other
693 stars 183 forks source link

Are the MBA parameters configured for each core or for each COS? #185

Closed dxlet closed 3 years ago

dxlet commented 3 years ago

Hello everyone!

MBA bandwidth limits per-CLOS are specified as a value in the range of zero to a maximum supported level of throttling for the platform

I learn from some documents that MBA can limit the bandwidth for each COLS, but in my experiment, MBA limits the bandwidth for each core not COLS.

The maxium memory bandwidth of my platform is about 40GB/s which is tested by mlc appliction

Firstly, I run stream on 10 cores without any rdt configuration

OMP_NUM_THREADS=10 ./stream_c.exe &
taskset -acp 0-9 pid

image

This application spent a total of about 20GB/s of bandwidth (50% of maxium bandwidth). Then，I constrained 0-9 cores to only use 10% of the memory bandwidth, but cannot limit their memory bandwidth.

pqos -e "mba:1=10"
pqos -a "core:1=0-9"

image

Secondly, I run stream on 3 cores without any rdt configuration. Only when the bandwidth of each core is large enough, my constraints can be reflected. image

pqos -e "mba:1=10"
pqos -a "core:1=1-10"

image

From this experiment, the MBA parameter is to constrain the memory bandwidth of each core rather than each CLOS.

I also tried the -I option to use the OS interface, and also tried to use -p to specify the pid to allocate, and I got the same result. Is the result as expected? Or is my operation wrong?

My platform information： Kernel: 5.11.16 OS: centos-release-7-9.2009 CPU: Intel Xeon gold 6278C.

kmabbasi commented 3 years ago

This is expected behavior. The MBA delay values are per CLOS. That delay value will be applied to every core associated with that CLOS. In your case, looks like 10% bandwidth generates approximately 2GB/s. So for 10 cores, its 20G.

dxlet commented 3 years ago

Thanks! I want to restrict the memory bandwidth more strictly (e.g. 1GB/s). Is there a way to achieve this goal? Even bypass pqos or resctrl, directly write MSR.

kmabbasi commented 3 years ago

Yes, you could use MBA SW controller. https://github.com/intel/intel-cmt-cat/wiki/MBM-MBA-how-to-guide#mba---mbm---configure-mba-in-mbs

Thanks, Khawar

dxlet commented 3 years ago

I tried MBA SW controller. It can limit the memory bandwidth to 2600MB/s at least, and it cannot limit the memory bandwidth to a lower value. Is this normal？

kmabbasi commented 3 years ago

SW controller allows you to provide values in MB/s. It doesn't provide more throttling than MBA values in % does.