Help with /sys/fs/resctrl?

[danbeimborn]

Hi all, I'm trying to get the /sys/fs/resctrl interface working with intel-cmt-cat 4.1.0 and kernel 5.9.11. I'm using the kernel-ml specfile and modifying the config.

I took the stock config and tried adding this line first: CONFIG_X86_CPU_RESCTRL=y

This by itself creates the /sys/fs/resctrl mount, but read-only- I'm not able to create subdirs to define/apply policies.

Please can you tell me the other options that need to be disabled or enabled for the /sys/fs/resctrl mount point to be read/write and able to set L3 cache policies etc? At the moment I'm able to get what I need with MSR build, though I'd prefer to use the sys/fs method if possible.

[kmabbasi]

Hi,

You probably need to enable RDT feature in Linux kernel. https://github.com/intel/intel-cmt-cat/wiki#enabling-intelr-rdt-support-in-the-linux-kernel-os-interface

If that doesn't work, could you please provide details of your environment which would be useful e.g. OS, platform, etc.

Thanks, Khawar

[danbeimborn]

Thank you Khawar, I was trying with kernel 5.9.11, I can't see any config options containing the string RDT.. is 5.9 too new for this to work?

I took the kernel-ml source rpm and modified the config file to enable CONFIG_X86_CPU_RESCTRL=y, but that was all I could find that appears to be related or needed.

Thank you for your help

[danbeimborn]

This is the help note under CONFIG_X86_CPU_RESCTRL on 5.9.11..

[kmabbasi]

Did you enable RDT feature by passing kernel command line parameters at boot? https://github.com/intel/intel-cmt-cat/wiki#enabling-intelr-rdt-support-in-the-linux-kernel-os-interface

What's the output of 'cat /proc/cmdline'? Also please share details of your environment.

Thanks, Khawar

[danbeimborn]

Ok here's direct output:

[root@ld4-hpg10-01 ~]# mount -t resctrl resctrl /sys/fs/resctrl/ [root@ld4-hpg10-01 ~]# cd /sys/fs/resctrl/ [root@ld4-hpg10-01 /sys/fs/resctrl]# ll total 0 -rw-r--r-- 1 root root 0 Jul 15 10:19 cpus -rw-r--r-- 1 root root 0 Jul 15 10:19 cpus_list dr-xr-xr-x 5 root root 0 Jul 15 10:19 info -rw-r--r-- 1 root root 0 Jul 15 10:19 mode dr-xr-xr-x 4 root root 0 Jul 15 10:19 mon_data dr-xr-xr-x 2 root root 0 Jul 15 10:19 mon_groups -rw-r--r-- 1 root root 0 Jul 15 10:19 schemata -r--r--r-- 1 root root 0 Jul 15 10:19 size -rw-r--r-- 1 root root 0 Jul 15 10:19 tasks [root@ld4-hpg10-01 /sys/fs/resctrl]# ls -ld . dr-xr-xr-x 5 root root 0 Jul 15 10:19 . [root@ld4-hpg10-01 /sys/fs/resctrl]# touch p0 touch: cannot touch ‘p0’: Permission denied

[root@ld4-hpg10-01 /sys/fs/resctrl]# cat /proc/cmdline BOOT_IMAGE=/vmlinuz-5.9.11-3.el7.x86_64 root=/dev/mapper/root-root ro crashkernel=auto norhgb spectre_v2=off nopti intel_idle.max_cstate=1 processor.max_cstate=1 nosoftlockup hpet=disable idle=poll mce=ignore_ce audit=0 edd=off ipv6.disable=1 mitigations=off nohz_full=1-17,19-35 rcu_nocbs=1-17,19-35 cgroup_disable=memory rd.lvm.lv=root/root rd.lvm.lv=root/swap rd.lvm.lv=root/usr noht msr.allow_writes=on skew_tick=1 rdt=cmt,mbmtotal,mbmlocal,l3cat,l3cdp,l2cat,l2cdp,mba quiet

[root@ld4-hpg10-01 /sys/fs/resctrl]# uname -r 5.9.11-3.el7.x86_64

[root@ld4-hpg10-01 /sys/fs/resctrl]# cat /etc/*release CentOS Linux release 7.9.2009 (Core) NAME="CentOS Linux" VERSION="7 (Core)" ID="centos" ID_LIKE="rhel fedora" VERSION_ID="7" PRETTY_NAME="CentOS Linux 7 (Core)" ANSI_COLOR="0;31" CPE_NAME="cpe:/o:centos:centos:7" HOME_URL="https://www.centos.org/" BUG_REPORT_URL="https://bugs.centos.org/"

CENTOS_MANTISBT_PROJECT="CentOS-7" CENTOS_MANTISBT_PROJECT_VERSION="7" REDHAT_SUPPORT_PRODUCT="centos" REDHAT_SUPPORT_PRODUCT_VERSION="7"

CentOS Linux release 7.9.2009 (Core) CentOS Linux release 7.9.2009 (Core)

Attached also the kernel config I used. the -3 on the kernel-ml is just my own release, with changes only to this attached file: config-5.9.11-3.el7.x86_64.txt

[kmabbasi]

Hi,

You can't use touch to create file within resctrl filesystem. You can use mkdir and rmdir for creating & deleting groups within resctrl respectively.

It would be more convenient to use pqos tool with '-I' option.

Thanks, Khawar

[danbeimborn]

Thank you Khawar, I can see I was just making some silly assumptions.

using pqos -I to set policies is correctly reflected in the /sys/fs tree. I see that it creates COS1-COS8 directories by itself, but the cpus lists and schemata update correctly as expected when I use pqos -I [-e|-a].

[kmabbasi]

Sounds good! I am going to close this issue now.

Thanks, Khawar