Add ENV_CPUS_ASSIGNED_MASK env variable

[LilyaBu]

Some applications expect a mask of cpus on which they isolate themselves (pktgen, testpmd). For the convenience of running such applications in cmk containers, it is useful to add allocated CPUs to the ENV, not only in the list, but also in the form of a mask.

[lmdaly]

Since most of the applications that consume the the ENV VARs do so through a script, would it be possible to convert the CPU IDs to a mask within such a script?

[LilyaBu]

Manipulations with bits are gentle in python, and not so nice in script. Besides it is common practice to put mask of cpus near list of cpus. For example in /sys/devices/system/node/node0 cpulist and cpumask

[anvien0111]

Hi @LilyaBu @lmdaly did you success using ENV_CPUS_ASSIGNED_MASK, I have problem with it like bellow: I want to isolate a task into 4 thread 2,38,3,39 The ENV_CPUS_ASSIGNED_MASK run to C00000000C, it True But when I using top 1 only CPU2 has 100% and notthing run into Cpu 3 38 39 Do you have any idea about this. Thanks

[przemeklal]

Hi @anvien0111, thanks for using CMK.

Most likely your application doesn't use all allocated cores - this may happen if this is some kind of DPDK based application and there's only one packet polling thread there because e.g. there's only one network interface/port handled in that application, or the application has some extra args for pairing interfaces/CPUs. CMK only allocates the cores, it's up to the application to actually use them.

[anvien0111]

Hi @przemeklal. Thanks for your answer, I have figure out I have problem with PythonLib psutil in here: https://github.com/intel/CPU-Manager-for-Kubernetes/blob/master/intel/isolate.py#116

when I set isolcpus=2-17,38-53 in boot/grup.cfg then using psutil.Process().cpu_affinity([2,3,4,5]) It not work, the process only bind into core 2 not all 4 core. If I remove isolcpu in boot/grub file It work well 100%CPU in all core 2,3,4,5 Do you have any idea about that or I need to set special config to my Pod to make it work. Thanks

[anvien0111]

@przemeklal when I set isolcpus=2-17,38-53 these Cpus would'nt show in command print(psutil.Process().cpu_affinity()). Is this make some problem when I want to affinity more than 1 core into 1 process??

[przemeklal]

Interesting findings. As a workaround you can add --no-affinity argument to cmk isolate and wrap your application command with taskset instead (passing either CPU list or the mask) and see if that works.

@phillyb7 - I'm tagging you here as it may be a valid bug

[anvien0111]

@przemeklal @phillyb7 Sorry but I test taskset not work too. Because when I set isolcpus=2-17, 38-53 then I cannot use taskset to bind multi cores in this range to one tasks. https://serverfault.com/questions/573025/taskset-not-working-over-a-range-of-cores-in-isolcpus Thanks

[lamtov]

@przemeklal @phillyb7 I check my docker info and kubelet is using 'Cgroup Driver: systemd' Do I need to change this setting to cgroupfs to make taskset work in Pod?

[lamtov]

@przemeklal @phillyb7 The problem is conflict of isolcpu and taskset. I change to use systemd-Cpuaffinity then taskset and your code work well. Thanks

[przemeklal]

@lamto20132223 great! And thanks for letting us now about the solution :)