intel / intel-cmt-cat

User space software for Intel(R) Resource Director Technology
http://www.intel.com/content/www/us/en/architecture-and-technology/resource-director-technology.html
Other
691 stars 181 forks source link

How is LLC occupancy obtained? #278

Closed wangerforcs closed 1 month ago

wangerforcs commented 1 month ago

My processor is an Intel(R) Xeon(R) Platinum 8269CY CPU with an L3 cache size of 35MB. I've noticed that this metric LLC occupancy is a multiple of 104KB, and in many cases, there's a significant discrepancy between the size calculated based on the cache ways allocated by CAT (Cache Allocation Technology) and the actual data monitored. Therefore, I would like to gain a deeper understanding of how this data is derived.

rkanagar commented 1 month ago

Hi wangerforcs, The perf_monitoring.c has all the code of monitoring by perf. It uses standard system call perf_event_open which is defined in lib/perf.c

https://www.systutorials.com/docs/linux/man/2-perf_event_open/ https://github.com/intel/intel-cmt-cat/blob/master/lib/perf_monitoring.c

Events:

    /** List of non virtual events */
    const enum pqos_mon_event mon_event[] = {
        PQOS_MON_EVENT_L3_OCCUP,
        PQOS_MON_EVENT_LMEM_BW,
        PQOS_MON_EVENT_TMEM_BW,
        PQOS_PERF_EVENT_LLC_MISS,
        PQOS_PERF_EVENT_LLC_REF,
        (enum pqos_mon_event)PQOS_PERF_EVENT_CYCLES,
        (enum pqos_mon_event)PQOS_PERF_EVENT_INSTRUCTIONS,
        PQOS_PERF_EVENT_LLC_MISS_PCIE_READ,
        PQOS_PERF_EVENT_LLC_MISS_PCIE_WRITE,
        PQOS_PERF_EVENT_LLC_REF_PCIE_READ,
        PQOS_PERF_EVENT_LLC_REF_PCIE_WRITE};

In the above events, the required event is monitored every second.

wangerforcs commented 1 month ago

Hi wangerforcs, The perf_monitoring.c has all the code of monitoring by perf. It uses standard system call perf_event_open which is defined in lib/perf.c

https://www.systutorials.com/docs/linux/man/2-perf_event_open/ https://github.com/intel/intel-cmt-cat/blob/master/lib/perf_monitoring.c

Events:

    /** List of non virtual events */
    const enum pqos_mon_event mon_event[] = {
        PQOS_MON_EVENT_L3_OCCUP,
        PQOS_MON_EVENT_LMEM_BW,
        PQOS_MON_EVENT_TMEM_BW,
        PQOS_PERF_EVENT_LLC_MISS,
        PQOS_PERF_EVENT_LLC_REF,
        (enum pqos_mon_event)PQOS_PERF_EVENT_CYCLES,
        (enum pqos_mon_event)PQOS_PERF_EVENT_INSTRUCTIONS,
        PQOS_PERF_EVENT_LLC_MISS_PCIE_READ,
        PQOS_PERF_EVENT_LLC_MISS_PCIE_WRITE,
        PQOS_PERF_EVENT_LLC_REF_PCIE_READ,
        PQOS_PERF_EVENT_LLC_REF_PCIE_WRITE};

In the above events, the required event is monitored every second.

Hi rkanagar. I already know it uses standard system call perf_event_open to record these metrics. However, I would like to know how this occupancy metric is caculated by system and why it is a multiple of 104KB.

rkanagar commented 1 month ago

Hi @wangerforcs , Please provide some example output you got. The pqos tool does not calculate anything in multiples of 104KB.

wangerforcs commented 1 month ago

Hi @wangerforcs , Please provide some example output you got. The pqos tool does not calculate anything in multiples of 104KB.

Hi rkanagar, some examples are as follows,all LLC are a multiple of 104KB. Why?

TIME 2024-07-15 15:34:31 CORE IPC MISSES LLC[KB] MBL[MB/s] MBR[MB/s] 0 0.50 194k 728.0 31.8 0.0 1 0.23 79k 0.0 27.8 0.0 2 0.18 8k 104.0 0.8 0.0 3 0.16 7k 208.0 0.9 0.0 4 3.92 3104k 0.0 0.1 0.0 5 0.19 16k 0.0 0.4 0.0 6 0.36 13k 0.0 1.5 0.0 7 1.18 60k 0.0 12.9 0.0 8 3.92 3011k 0.0 0.2 0.0 9 0.36 251k 104.0 26.9 0.0 10 0.29 48k 208.0 4.5 0.0 11 0.11 65k 0.0 0.9 0.0 12 0.20 11k 0.0 1.2 0.0 13 0.17 8k 0.0 0.5 0.0 14 0.37 79k 312.0 6.4 0.0 15 0.37 34k 208.0 2.7 0.0 16 0.37 45k 208.0 5.4 0.0 17 0.32 79k 0.0 5.9 0.0 18 0.36 13k 104.0 1.1 0.0 19 0.42 26k 0.0 4.8 0.0 20 1.24 94k 0.0 21.6 0.0 21 4.07 876k 0.0 0.0 0.0 22 0.16 14k 104.0 0.9 0.0 23 0.32 3k 104.0 1.8 0.0 24 3.12 4213k 104.0 0.0 0.0 25 3.13 1175k 0.0 0.0 0.0 26 0.34 41k 312.0 1.6 0.0 27 0.61 18k 0.0 6.8 0.0 28 2.20 944k 0.0 0.0 0.0 29 0.15 55k 208.0 7.1 0.0 30 0.48 15k 0.0 0.1 0.0 31 0.41 46k 208.0 2.6 0.0 32 0.51 59k 0.0 9.8 0.0 33 0.27 49k 104.0 11.5 0.0^C

rkanagar commented 1 month ago

HI @wangerforcs , I am unable to help you in this issue. I dont see any pqos tool issue. I dont know how the Linux system call perf_event_open calculates this value.

wangerforcs commented 1 month ago

HI @wangerforcs , I am unable to help you in this issue. I dont see any pqos tool issue. I dont know how the Linux system call perf_event_open calculates this value.

Thanks for your response.