search

intel / PerfSpect

System performance characterization tool based on linux perf
https://intel.github.io/PerfSpect/
BSD 3-Clause "New" or "Revised" License
332 stars 30 forks source link

metric_NUMA local DRAM + metric_NUMA remote DRAM != 100% #37

Closed jasonneverstop closed 1 year ago

jasonneverstop commented 1 year ago

skylake centos7 perfspect-1.2.11 Why is the sum of 'metric_NUMA %_Reads addressed to local DRAM' and metric_NUMA %_Reads addressed to remote DRAM not equal to 100%?

jasonneverstop commented 1 year ago

And the value of the UNC_CHA_TOR_INSERTS.IA_MISS.0x40431.x collected by perf is often 0

hilldani commented 1 year ago
I ran perfspect 1.2.11 on centos 7 (ami-08c191625cfb7ee61), SKX (m5.metal) and remote and local DRAM access added up to 100% metric avg
metric_NUMA %_Reads addressed to local DRAM 90.25741337
metric_NUMA %_Reads addressed to remote DRAM 9.74258663

I ran perfspect with verbose flag -v

sudo ./perf-collect -v -t 10
./perf-postprocess -v

and their combined output was

2023-06-02 21:04:40,102 INFO: nmi_watchdog is temporary disabled. Will enable after collection.
2023-06-02 21:04:40,107 INFO: Run mode: system
2023-06-02 21:04:41,816 WARNING: Perf unsupported events not counted: ['cstate_core/c6-residency/;', 'cstate_pkg/c6-residency/;']
2023-06-02 21:04:41,916 INFO: PMUs not in use
2023-06-02 21:04:41,916 INFO: perf stat -I 5000 -x , -a -o perfstat.csv -e '{cpu/event=0x28,umask=0x07,period=200003,name='CORE_POWER.LVL0_TURBO_LICENSE'/,cpu/event=0x28,umask=0x18,period=200003,name='CORE_POWER.LVL1_TURBO_LICENSE'/,cpu/event=0x28,umask=0x20,period=200003,name='CORE_POWER.LVL2_TURBO_LICENSE'/,cpu/event=0x51,umask=0x01,period=2000003,name='L1D.REPLACEMENT'/,cpu-cycles,ref-cycles,instructions}','{cpu/event=0xc3,umask=0x01,edge,period=100003,name='MACHINE_CLEARS.COUNT'/,cpu/event=0xc5,umask=0x00,period=[400](https://github.com/intel-innersource/applications.analyzers.cloudcompute.perfspect/actions/runs/5159695458/jobs/9294968350#step:6:401)009,name='BR_MISP_RETIRED.ALL_BRANCHES'/,cpu/event=0x0d,umask=0x80,period=2000003,name='INT_MISC.CLEAR_RESTEER_CYCLES'/,cpu/event=0xd1,umask=0x01,period=2000003,name='MEM_LOAD_RETIRED.L1_HIT'/,cpu-cycles,ref-cycles,instructions}','{cpu/event=0x0e,umask=0x01,period=2000003,name='UOPS_ISSUED.ANY'/,cpu/event=0xc2,umask=0x02,period=2000003,name='UOPS_RETIRED.RETIRE_SLOTS'/,cpu/event=0x0d,umask=0x01,period=2000003,name='INT_MISC.RECOVERY_CYCLES_ANY'/,cpu/event=0x0d,umask=0x01,period=2000003,name='INT_MISC.RECOVERY_CYCLES'/}','{cpu-cycles,ref-cycles,instructions}','{cpu/event=0x3c,umask=0x0,period=2000003,name='CPU_CLK_UNHALTED.THREAD_ANY'/,cpu/event=0x9c,umask=0x01,period=2000003,name='IDQ_UOPS_NOT_DELIVERED.CORE'/,cpu/event=0xa3,umask=0x14,cmask=0x14,period=2000003,name='CYCLE_ACTIVITY.STALLS_MEM_ANY'/,cpu/event=0xa3,umask=0x04,cmask=0x04,period=2000003,name='CYCLE_ACTIVITY.STALLS_TOTAL'/,cpu-cycles,ref-cycles,instructions}','{cpu/event=0xa6,umask=0x02,period=2000003,name='EXE_ACTIVITY.1_PORTS_UTIL'/,cpu/event=0xa6,umask=0x04,period=2000003,name='EXE_ACTIVITY.2_PORTS_UTIL'/,cpu/event=0xa6,umask=0x40,period=2000003,name='EXE_ACTIVITY.BOUND_ON_STORES'/,cpu/event=0xa6,umask=0x01,period=2000003,name='EXE_ACTIVITY.EXE_BOUND_0_PORTS'/,cpu-cycles,ref-cycles,instructions}','{cpu/event=0xd1,umask=0x02,period=100003,name='MEM_LOAD_RETIRED.L2_HIT'/,cpu/event=0xd1,umask=0x40,period=100007,name='MEM_LOAD_RETIRED.FB_HIT'/,cpu/event=0xd1,umask=0x08,period=100003,name='MEM_LOAD_RETIRED.L1_MISS'/,cpu/event=0x48,umask=0x02,cmask=0x01,period=2000003,name='L1D_PEND_MISS.FB_FULL:c1'/,cpu-cycles,ref-cycles,instructions}','{cpu/event=0xa3,umask=0x0c,cmask=0x0c,period=2000003,name='CYCLE_ACTIVITY.STALLS_L1D_MISS'/,cpu/event=0xa3,umask=0x05,cmask=0x05,period=2000003,name='CYCLE_ACTIVITY.STALLS_L2_MISS'/,cpu/event=0xa3,umask=0x06,cmask=0x06,period=2000003,name='CYCLE_ACTIVITY.STALLS_L3_MISS'/,cpu/event=0x60,umask=0x04,cmask=0x01,period=200003,name='OFFCORE_REQUESTS_OUTSTANDING.CYCLES_WITH_DEMAND_RFO'/,cpu-cycles,ref-cycles,instructions}','{cpu/event=0x85,umask=0x0e,period=100003,name='ITLB_MISSES.WALK_COMPLETED'/,cpu/event=0x85,umask=0x10,period=100003,name='ITLB_MISSES.WALK_ACTIVE'/,cpu/event=0x08,umask=0x0e,period=100003,name='DTLB_LOAD_MISSES.WALK_COMPLETED'/,cpu/event=0x08,umask=0x10,cmask=0x01,period=100003,name='DTLB_LOAD_MISSES.WALK_ACTIVE'/,cpu-cycles,ref-cycles,instructions}','{cpu/event=0x49,umask=0x0e,period=100003,name='DTLB_STORE_MISSES.WALK_COMPLETED'/,cpu/event=0x49,umask=0x10,period=100003,name='DTLB_STORE_MISSES.WALK_ACTIVE'/,cpu/event=0x9c,umask=0x01,cmask=0x4,period=2000003,name='IDQ_UOPS_NOT_DELIVERED.CYCLES_0_UOPS_DELIV.CORE'/,cpu/event=0xe6,umask=0x01,period=100003,name='BACLEARS.ANY'/,cpu-cycles,ref-cycles,instructions}','{cpu/event=0xd0,umask=0x21,cmask=0x01,period=100007,name='MEM_INST_RETIRED.LOCK_LOADS'/,cpu/event=0x24,umask=0xe2,cmask=0x00,period=200003,name='L2_RQSTS.ALL_RFO'/,cpu/event=0xd0,umask=0x82,cmask=0x00,period=200003,name='MEM_INST_RETIRED.ALL_STORES'/,cpu/event=0x24,umask=0xc2,cmask=0x00,period=200003,name='L2_RQSTS.RFO_HIT'/,cpu-cycles,ref-cycles,instructions}','{cpu/event=0xb7,umask=0x01,offcore_rsp=0x10003C0001,period=100003,name='OCR.DEMAND_DATA_RD.L3_HIT.HITM_OTHER_CORE'/,cpu/event=0xb7,umask=0x01,offcore_rsp=0x8003C0001,period=100003,name='OCR.DEMAND_DATA_RD.L3_HIT.HIT_OTHER_CORE_FWD'/,cpu/event=0x60,umask=0x10,cmask=0x06,period=2000003,name='OFFCORE_REQUESTS_OUTSTANDING.L3_MISS_DEMAND_DATA_RD_GE_6'/,cpu/event=0x60,umask=0x10,cmask=0x01,period=2000003,name='OFFCORE_REQUESTS_OUTSTANDING.CYCLES_WITH_L3_MISS_DEMAND_DATA_RD'/,cpu-cycles,ref-cycles,instructions}','{cpu/event=0xb1,umask=0x02,cmask=0x1,period=2000003,name='UOPS_EXECUTED.CORE_CYCLES_GE_1'/,cpu/event=0xb1,umask=0x02,cmask=0x2,period=2000003,name='UOPS_EXECUTED.CORE_CYCLES_GE_2'/,cpu/event=0xb1,umask=0x02,cmask=0x3,period=2000003,name='UOPS_EXECUTED.CORE_CYCLES_GE_3'/,cpu/event=0xc2,umask=0x04,cmask=0x00,period=2000003,name='UOPS_RETIRED.MACRO_FUSED'/,cpu-cycles,ref-cycles,instructions}','{cpu/event=0xc7,umask=0x03,cmask=0x00,period=2000003,name='FP_ARITH_INST_RETIRED.SCALAR_SINGLE:u0x03'/,cpu/event=0xc7,umask=0xfc,cmask=0x00,period=2000003,name='FP_ARITH_INST_RETIRED.128B_PACKED_DOUBLE:u0xfc'/,cpu/event=0x80,umask=0x4,name='ICACHE_16B.IFDATA_STALL'/,cpu/event=0x80,umask=0x4,cmask=0x1,edge=0x1,name='ICACHE_16B.IFDATA_STALL:c1:e1'/,cpu-cycles,ref-cycles,instructions}','{cpu/event=0x24,umask=0xe4,period=200003,name='L2_RQSTS.ALL_CODE_RD'/,cpu/event=0x85,umask=0x04,period=100003,name='ITLB_MISSES.WALK_COMPLETED_2M_4M'/,cpu/event=0xf1,umask=0x1f,period=100003,name='L2_LINES_IN.ALL'/,cpu/event=0xd1,umask=0x10,period=50021,name='MEM_LOAD_RETIRED.L2_MISS'/,cpu-cycles,ref-cycles,instructions}','{cpu/event=0x24,umask=0x24,period=200003,name='L2_RQSTS.CODE_RD_MISS'/,cpu/event=0x08,umask=0x04,period=2000003,name='DTLB_LOAD_MISSES.WALK_COMPLETED_2M_4M'/,cpu/event=0x08,umask=0x02,period=2000003,name='DTLB_LOAD_MISSES.WALK_COMPLETED_4K'/,cpu/event=0x08,umask=0x08,period=2000003,name='DTLB_LOAD_MISSES.WALK_COMPLETED_1G'/,cpu-cycles,ref-cycles,instructions}','{cpu/event=0xb0,umask=0x10,period=100003,name='OFFCORE_REQUESTS.L3_MISS_DEMAND_DATA_RD'/,cpu/event=0xd1,umask=0x40,period=100007,name='MEM_LOAD_RETIRED.FB_HIT'/,cpu/event=0xa3,umask=0x10,cmask=0x16,period=2000003,name='CYCLE_ACTIVITY.CYCLES_MEM_ANY'/,cpu/event=0xa3,umask=0x08,cmask=0x08,period=2000003,name='CYCLE_ACTIVITY.CYCLES_L1D_MISS'/,cpu-cycles,ref-cycles,instructions}','{cpu/event=0x79,umask=0x30,period=2000003,name='IDQ.MS_UOPS'/,cpu/event=0x60,umask=0x10,period=2000003,name='OFFCORE_REQUESTS_OUTSTANDING.L3_MISS_DEMAND_DATA_RD'/,cpu/event=0x83,umask=0x04,period=200003,name='ICACHE_64B.IFTAG_STALL'/,cpu/event=0x08,umask=0x20,cmask=0x01,period=2000003,name='DTLB_LOAD_MISSES.STLB_HIT:c1'/,cpu-cycles,ref-cycles,instructions}','{cpu/event=0x80,umask=0x4,name='ICACHE_16B.IFDATA_STALL'/,cpu/event=0x80,umask=0x4,cmask=0x1,edge=0x1,name='ICACHE_16B.IFDATA_STALL:c1:e1'/,cpu/event=0x14,umask=0x01,period=2000003,name='ARITH.DIVIDER_ACTIVE'/,cpu/event=0xb1,umask=0x02,inv=0x1,cmask=0x1,period=2000003,name='UOPS_EXECUTED.CORE_CYCLES_NONE'/,cpu-cycles,ref-cycles,instructions}','{cpu/event=0x3c,umask=0x2,name='CPU_CLK_THREAD_UNHALTED.ONE_THREAD_ACTIVE'/,cpu/event=0x3c,umask=0x1,name='CPU_CLK_THREAD_UNHALTED.REF_XCLK_ANY'/,cpu-cycles:k,ref-cycles:k,instructions:k}','{cpu/event=0x79,umask=0x24,cmask=0x01,period=2000003,name='IDQ.ALL_MITE_CYCLES_ANY_UOPS'/,cpu/event=0x79,umask=0x24,cmask=0x04,period=2000003,name='IDQ.ALL_MITE_CYCLES_4_UOPS'/,cpu/event=0xb7,umask=0x01,offcore_rsp=0x10003C0002,period=100003,name='OCR.DEMAND_RFO.L3_HIT.HITM_OTHER_CORE'/,cpu/event=0xb7,umask=0x01,offcore_rsp=0x10003C0020,period=100003,name='OCR.PF_L2_RFO.L3_HIT.HITM_OTHER_CORE'/,cpu-cycles,ref-cycles,instructions}','{cpu/event=0x79,umask=0x18,cmask=0x01,period=2000003,name='IDQ.ALL_DSB_CYCLES_ANY_UOPS'/,cpu/event=0x79,umask=0x18,cmask=0x04,period=2000003,name='IDQ.ALL_DSB_CYCLES_4_UOPS'/,cpu/event=0xb7,umask=0x01,offcore_rsp=0x103FC00002,period=100003,name='OCR.DEMAND_RFO.L3_MISS.REMOTE_HITM'/,cpu/event=0xb7,umask=0x01,offcore_rsp=0x103FC00020,period=100003,name='OCR.PF_L2_RFO.L3_MISS.REMOTE_HITM'/,cpu-cycles,ref-cycles,instructions}','{cpu/event=0xd2,umask=0x02,cmask=0x00,period=20011,name='MEM_LOAD_L3_HIT_RETIRED.XSNP_HIT'/,cpu/event=0xd2,umask=0x04,cmask=0x00,period=20011,name='MEM_LOAD_L3_HIT_RETIRED.XSNP_HITM'/,cpu/event=0xb7,umask=0x01,offcore_rsp=0x3F840007F7,name='OCR.ALL_READS.L3_MISS_LOCAL_DRAM.ANY_SNOOP'/,cpu/event=0xb7,umask=0x01,offcore_rsp=0x3FB80007F7,name='OCR.ALL_READS.L3_MISS_LOCAL_DRAM.ANY_SNOOP_ocr_msr_3fB80007f7'/}','{cpu-cycles,ref-cycles,instructions}','{cpu/event=0xb1,umask=0x10,cmask=0x00,period=2000003,name='UOPS_EXECUTED.X87'/,cpu/event=0xb1,umask=0x01,cmask=0x00,period=2000003,name='UOPS_EXECUTED.THREAD'/,cpu/event=0xb7,umask=0x01,offcore_rsp=0x103FC007F7,name='OCR.ALL_READS.L3_MISS.REMOTE_HITM'/,cpu/event=0xb7,umask=0x01,offcore_rsp=0x083FC007F7,name='OCR.ALL_READS.L3_MISS.REMOTE_HIT_FORWARD'/}','{cpu-cycles,ref-cycles,instructions}','{power/energy-pkg/,power/energy-ram/}','{uncore_imc_0/event=0x04,umask=0x03,name='UNC_M_CAS_COUNT.RD.0'/,uncore_imc_0/event=0x04,umask=0x0c,name='UNC_M_CAS_COUNT.WR.0'/}','{uncore_imc_1/event=0x04,umask=0x03,name='UNC_M_CAS_COUNT.RD.1'/,uncore_imc_1/event=0x04,umask=0x0c,name='UNC_M_CAS_COUNT.WR.1'/}','{uncore_imc_2/event=0x04,umask=0x03,name='UNC_M_CAS_COUNT.RD.2'/,uncore_imc_2/event=0x04,umask=0x0c,name='UNC_M_CAS_COUNT.WR.2'/}','{uncore_imc_3/event=0x04,umask=0x03,name='UNC_M_CAS_COUNT.RD.3'/,uncore_imc_3/event=0x04,umask=0x0c,name='UNC_M_CAS_COUNT.WR.3'/}','{uncore_imc_4/event=0x04,umask=0x03,name='UNC_M_CAS_COUNT.RD.4'/,uncore_imc_4/event=0x04,umask=0x0c,name='UNC_M_CAS_COUNT.WR.4'/}','{uncore_imc_5/event=0x04,umask=0x03,name='UNC_M_CAS_COUNT.RD.5'/,uncore_imc_5/event=0x04,umask=0x0c,name='UNC_M_CAS_COUNT.WR.5'/}','{uncore_cha_0/event=0x0,umask=0x0,name='UNC_CHA_CLOCKTICKS.0'/,uncore_cha_0/event=0x36,umask=0x21,config1=0x[404](https://github.com/intel-innersource/applications.analyzers.cloudcompute.perfspect/actions/runs/5159695458/jobs/9294968350#step:6:405)3300000000,name='UNC_CHA_TOR_OCCUPANCY.IA_MISS.0x40433.0'/,uncore_cha_0/event=0x35,umask=0x21,config1=0x4043300000000,name='UNC_CHA_TOR_INSERTS.IA_MISS.0x40433.0'/}','{uncore_cha_1/event=0x0,umask=0x0,name='UNC_CHA_CLOCKTICKS.1'/,uncore_cha_1/event=0x36,umask=0x21,config1=0x4043300000000,name='UNC_CHA_TOR_OCCUPANCY.IA_MISS.0x40433.1'/,uncore_cha_1/event=0x35,umask=0x21,config1=0x4043300000000,name='UNC_CHA_TOR_INSERTS.IA_MISS.0x40433.1'/}','{uncore_cha_2/event=0x0,umask=0x0,name='UNC_CHA_CLOCKTICKS.2'/,uncore_cha_2/event=0x36,umask=0x21,config1=0x4043300000000,name='UNC_CHA_TOR_OCCUPANCY.IA_MISS.0x40433.2'/,uncore_cha_2/event=0x35,umask=0x21,config1=0x4043300000000,name='UNC_CHA_TOR_INSERTS.IA_MISS.0x40433.2'/}','{uncore_cha_3/event=0x0,umask=0x0,name='UNC_CHA_CLOCKTICKS.3'/,uncore_cha_3/event=0x36,umask=0x21,config1=0x4043300000000,name='UNC_CHA_TOR_OCCUPANCY.IA_MISS.0x40433.3'/,uncore_cha_3/event=0x35,umask=0x21,config1=0x4043300000000,name='UNC_CHA_TOR_INSERTS.IA_MISS.0x40433.3'/}','{uncore_cha_4/event=0x0,umask=0x0,name='UNC_CHA_CLOCKTICKS.4'/,uncore_cha_4/event=0x36,umask=0x21,config1=0x4043300000000,name='UNC_CHA_TOR_OCCUPANCY.IA_MISS.0x40433.4'/,uncore_cha_4/event=0x35,umask=0x21,config1=0x4043300000000,name='UNC_CHA_TOR_INSERTS.IA_MISS.0x40433.4'/}','{uncore_cha_5/event=0x0,umask=0x0,name='UNC_CHA_CLOCKTICKS.5'/,uncore_cha_5/event=0x36,umask=0x21,config1=0x4043300000000,name='UNC_CHA_TOR_OCCUPANCY.IA_MISS.0x40433.5'/,uncore_cha_5/event=0x35,umask=0x21,config1=0x4043300000000,name='UNC_CHA_TOR_INSERTS.IA_MISS.0x40433.5'/}','{uncore_cha_6/event=0x0,umask=0x0,name='UNC_CHA_CLOCKTICKS.6'/,uncore_cha_6/event=0x36,umask=0x21,config1=0x4043300000000,name='UNC_CHA_TOR_OCCUPANCY.IA_MISS.0x40433.6'/,uncore_cha_6/event=0x35,umask=0x21,config1=0x4043300000000,name='UNC_CHA_TOR_INSERTS.IA_MISS.0x40433.6'/}','{uncore_cha_7/event=0x0,umask=0x0,name='UNC_CHA_CLOCKTICKS.7'/,uncore_cha_7/event=0x36,umask=0x21,config1=0x4043300000000,name='UNC_CHA_TOR_OCCUPANCY.IA_MISS.0x40433.7'/,uncore_cha_7/event=0x35,umask=0x21,config1=0x4043300000000,name='UNC_CHA_TOR_INSERTS.IA_MISS.0x40433.7'/}','{uncore_cha_8/event=0x0,umask=0x0,name='UNC_CHA_CLOCKTICKS.8'/,uncore_cha_8/event=0x36,umask=0x21,config1=0x4043300000000,name='UNC_CHA_TOR_OCCUPANCY.IA_MISS.0x40433.8'/,uncore_cha_8/event=0x35,umask=0x21,config1=0x4043300000000,name='UNC_CHA_TOR_INSERTS.IA_MISS.0x40433.8'/}','{uncore_cha_9/event=0x0,umask=0x0,name='UNC_CHA_CLOCKTICKS.9'/,uncore_cha_9/event=0x36,umask=0x21,config1=0x4043300000000,name='UNC_CHA_TOR_OCCUPANCY.IA_MISS.0x40433.9'/,uncore_cha_9/event=0x35,umask=0x21,config1=0x4043300000000,name='UNC_CHA_TOR_INSERTS.IA_MISS.0x40433.9'/}','{uncore_cha_10/event=0x0,umask=0x0,name='UNC_CHA_CLOCKTICKS.10'/,uncore_cha_10/event=0x36,umask=0x21,config1=0x4043300000000,name='UNC_CHA_TOR_OCCUPANCY.IA_MISS.0x40433.10'/,uncore_cha_10/event=0x35,umask=0x21,config1=0x4043300000000,name='UNC_CHA_TOR_INSERTS.IA_MISS.0x40433.10'/}','{uncore_cha_11/event=0x0,umask=0x0,name='UNC_CHA_CLOCKTICKS.11'/,uncore_cha_11/event=0x36,umask=0x21,config1=0x4043300000000,name='UNC_CHA_TOR_OCCUPANCY.IA_MISS.0x40433.11'/,uncore_cha_11/event=0x35,umask=0x21,config1=0x4043300000000,name='UNC_CHA_TOR_INSERTS.IA_MISS.0x40433.11'/}','{uncore_cha_12/event=0x0,umask=0x0,name='UNC_CHA_CLOCKTICKS.12'/,uncore_cha_12/event=0x36,umask=0x21,config1=0x4043300000000,name='UNC_CHA_TOR_OCCUPANCY.IA_MISS.0x40433.12'/,uncore_cha_12/event=0x35,umask=0x21,config1=0x4043300000000,name='UNC_CHA_TOR_INSERTS.IA_MISS.0x40433.12'/}','{uncore_cha_13/event=0x0,umask=0x0,name='UNC_CHA_CLOCKTICKS.13'/,uncore_cha_13/event=0x36,umask=0x21,config1=0x4043300000000,name='UNC_CHA_TOR_OCCUPANCY.IA_MISS.0x40433.13'/,uncore_cha_13/event=0x35,umask=0x21,config1=0x4043300000000,name='UNC_CHA_TOR_INSERTS.IA_MISS.0x40433.13'/}','{uncore_cha_14/event=0x0,umask=0x0,name='UNC_CHA_CLOCKTICKS.14'/,uncore_cha_14/event=0x36,umask=0x21,config1=0x4043300000000,name='UNC_CHA_TOR_OCCUPANCY.IA_MISS.0x40433.14'/,uncore_cha_14/event=0x35,umask=0x21,config1=0x4043300000000,name='UNC_CHA_TOR_INSERTS.IA_MISS.0x40433.14'/}','{uncore_cha_15/event=0x0,umask=0x0,name='UNC_CHA_CLOCKTICKS.15'/,uncore_cha_15/event=0x36,umask=0x21,config1=0x4043300000000,name='UNC_CHA_TOR_OCCUPANCY.IA_MISS.0x40433.15'/,uncore_cha_15/event=0x35,umask=0x21,config1=0x4043300000000,name='UNC_CHA_TOR_INSERTS.IA_MISS.0x40433.15'/}','{uncore_cha_16/event=0x0,umask=0x0,name='UNC_CHA_CLOCKTICKS.16'/,uncore_cha_16/event=0x36,umask=0x21,config1=0x4043300000000,name='UNC_CHA_TOR_OCCUPANCY.IA_MISS.0x40433.16'/,uncore_cha_16/event=0x35,umask=0x21,config1=0x4043300000000,name='UNC_CHA_TOR_INSERTS.IA_MISS.0x40433.16'/}','{uncore_cha_17/event=0x0,umask=0x0,name='UNC_CHA_CLOCKTICKS.17'/,uncore_cha_17/event=0x36,umask=0x21,config1=0x4043300000000,name='UNC_CHA_TOR_OCCUPANCY.IA_MISS.0x40433.17'/,uncore_cha_17/event=0x35,umask=0x21,config1=0x4043300000000,name='UNC_CHA_TOR_INSERTS.IA_MISS.0x40433.17'/}','{uncore_cha_18/event=0x0,umask=0x0,name='UNC_CHA_CLOCKTICKS.18'/,uncore_cha_18/event=0x36,umask=0x21,config1=0x4043300000000,name='UNC_CHA_TOR_OCCUPANCY.IA_MISS.0x40433.18'/,uncore_cha_18/event=0x35,umask=0x21,config1=0x4043300000000,name='UNC_CHA_TOR_INSERTS.IA_MISS.0x40433.18'/}','{uncore_cha_19/event=0x0,umask=0x0,name='UNC_CHA_CLOCKTICKS.19'/,uncore_cha_19/event=0x36,umask=0x21,config1=0x4043300000000,name='UNC_CHA_TOR_OCCUPANCY.IA_MISS.0x40433.19'/,uncore_cha_19/event=0x35,umask=0x21,config1=0x4043300000000,name='UNC_CHA_TOR_INSERTS.IA_MISS.0x40433.19'/}','{uncore_cha_20/event=0x0,umask=0x0,name='UNC_CHA_CLOCKTICKS.20'/,uncore_cha_20/event=0x36,umask=0x21,config1=0x4043300000000,name='UNC_CHA_TOR_OCCUPANCY.IA_MISS.0x40433.20'/,uncore_cha_20/event=0x35,umask=0x21,config1=0x4043300000000,name='UNC_CHA_TOR_INSERTS.IA_MISS.0x40433.20'/}','{uncore_cha_21/event=0x0,umask=0x0,name='UNC_CHA_CLOCKTICKS.21'/,uncore_cha_21/event=0x36,umask=0x21,config1=0x4043300000000,name='UNC_CHA_TOR_OCCUPANCY.IA_MISS.0x40433.21'/,uncore_cha_21/event=0x35,umask=0x21,config1=0x4043300000000,name='UNC_CHA_TOR_INSERTS.IA_MISS.0x40433.21'/}','{uncore_cha_22/event=0x0,umask=0x0,name='UNC_CHA_CLOCKTICKS.22'/,uncore_cha_22/event=0x36,umask=0x21,config1=0x4043300000000,name='UNC_CHA_TOR_OCCUPANCY.IA_MISS.0x40433.22'/,uncore_cha_22/event=0x35,umask=0x21,config1=0x4043300000000,name='UNC_CHA_TOR_INSERTS.IA_MISS.0x40433.22'/}','{uncore_cha_23/event=0x0,umask=0x0,name='UNC_CHA_CLOCKTICKS.23'/,uncore_cha_23/event=0x36,umask=0x21,config1=0x4043300000000,name='UNC_CHA_TOR_OCCUPANCY.IA_MISS.0x40433.23'/,uncore_cha_23/event=0x35,umask=0x21,config1=0x4043300000000,name='UNC_CHA_TOR_INSERTS.IA_MISS.0x40433.23'/}','{uncore_cha_24/event=0x0,umask=0x0,name='UNC_CHA_CLOCKTICKS.24'/,uncore_cha_24/event=0x36,umask=0x21,config1=0x4043300000000,name='UNC_CHA_TOR_OCCUPANCY.IA_MISS.0x40433.24'/,uncore_cha_24/event=0x35,umask=0x21,config1=0x4043300000000,name='UNC_CHA_TOR_INSERTS.IA_MISS.0x40433.24'/}','{uncore_cha_25/event=0x0,umask=0x0,name='UNC_CHA_CLOCKTICKS.25'/,uncore_cha_25/event=0x36,umask=0x21,config1=0x4043300000000,name='UNC_CHA_TOR_OCCUPANCY.IA_MISS.0x40433.25'/,uncore_cha_25/event=0x35,umask=0x21,config1=0x4043300000000,name='UNC_CHA_TOR_INSERTS.IA_MISS.0x40433.25'/}','{uncore_cha_0/event=0x35,umask=0x21,config1=0x4043200000000,name='UNC_CHA_TOR_INSERTS.IA_MISS.0x40432.0'/,uncore_cha_0/event=0x36,umask=0x21,config1=0x4043200000000,name='UNC_CHA_TOR_OCCUPANCY.IA_MISS.0x40432.0'/}','{uncore_cha_1/event=0x35,umask=0x21,config1=0x4043200000000,name='UNC_CHA_TOR_INSERTS.IA_MISS.0x40432.1'/,uncore_cha_1/event=0x36,umask=0x21,config1=0x4043200000000,name='UNC_CHA_TOR_OCCUPANCY.IA_MISS.0x40432.1'/}','{uncore_cha_2/event=0x35,umask=0x21,config1=0x4043200000000,name='UNC_CHA_TOR_INSERTS.IA_MISS.0x40432.2'/,uncore_cha_2/event=0x36,umask=0x21,config1=0x4043200000000,name='UNC_CHA_TOR_OCCUPANCY.IA_MISS.0x40432.2'/}','{uncore_cha_3/event=0x35,umask=0x21,config1=0x4043200000000,name='UNC_CHA_TOR_INSERTS.IA_MISS.0x40432.3'/,uncore_cha_3/event=0x36,umask=0x21,config1=0x4043200000000,name='UNC_CHA_TOR_OCCUPANCY.IA_MISS.0x40432.3'/}','{uncore_cha_4/event=0x35,umask=0x21,config1=0x4043200000000,name='UNC_CHA_TOR_INSERTS.IA_MISS.0x40432.4'/,uncore_cha_4/event=0x36,umask=0x21,config1=0x4043200000000,name='UNC_CHA_TOR_OCCUPANCY.IA_MISS.0x40432.4'/}','{uncore_cha_5/event=0x35,umask=0x21,config1=0x4043200000000,name='UNC_CHA_TOR_INSERTS.IA_MISS.0x40432.5'/,uncore_cha_5/event=0x36,umask=0x21,config1=0x4043200000000,name='UNC_CHA_TOR_OCCUPANCY.IA_MISS.0x40432.5'/}','{uncore_cha_6/event=0x35,umask=0x21,config1=0x4043200000000,name='UNC_CHA_TOR_INSERTS.IA_MISS.0x40432.6'/,uncore_cha_6/event=0x36,umask=0x21,config1=0x4043200000000,name='UNC_CHA_TOR_OCCUPANCY.IA_MISS.0x40432.6'/}','{uncore_cha_7/event=0x35,umask=0x21,config1=0x4043200000000,name='UNC_CHA_TOR_INSERTS.IA_MISS.0x40432.7'/,uncore_cha_7/event=0x36,umask=0x21,config1=0x4043200000000,name='UNC_CHA_TOR_OCCUPANCY.IA_MISS.0x40432.7'/}','{uncore_cha_8/event=0x35,umask=0x21,config1=0x4043200000000,name='UNC_CHA_TOR_INSERTS.IA_MISS.0x40432.8'/,uncore_cha_8/event=0x36,umask=0x21,config1=0x4043200000000,name='UNC_CHA_TOR_OCCUPANCY.IA_MISS.0x40432.8'/}','{uncore_cha_9/event=0x35,umask=0x21,config1=0x4043200000000,name='UNC_CHA_TOR_INSERTS.IA_MISS.0x40432.9'/,uncore_cha_9/event=0x36,umask=0x21,config1=0x4043200000000,name='UNC_CHA_TOR_OCCUPANCY.IA_MISS.0x40432.9'/}','{uncore_cha_10/event=0x35,umask=0x21,config1=0x4043200000000,name='UNC_CHA_TOR_INSERTS.IA_MISS.0x40432.10'/,uncore_cha_10/event=0x36,umask=0x21,config1=0x4043200000000,name='UNC_CHA_TOR_OCCUPANCY.IA_MISS.0x40432.10'/}','{uncore_cha_11/event=0x35,umask=0x21,config1=0x4043200000000,name='UNC_CHA_TOR_INSERTS.IA_MISS.0x40432.11'/,uncore_cha_11/event=0x36,umask=0x21,config1=0x4043200000000,name='UNC_CHA_TOR_OCCUPANCY.IA_MISS.0x40432.11'/}','{uncore_cha_12/event=0x35,umask=0x21,config1=0x4043200000000,name='UNC_CHA_TOR_INSERTS.IA_MISS.0x40432.12'/,uncore_cha_12/event=0x36,umask=0x21,config1=0x4043200000000,name='UNC_CHA_TOR_OCCUPANCY.IA_MISS.0x40432.12'/}','{uncore_cha_13/event=0x35,umask=0x21,config1=0x4043200000000,name='UNC_CHA_TOR_INSERTS.IA_MISS.0x40432.13'/,uncore_cha_13/event=0x36,umask=0x21,config1=0x4043200000000,name='UNC_CHA_TOR_OCCUPANCY.IA_MISS.0x40432.13'/}','{uncore_cha_14/event=0x35,umask=0x21,config1=0x4043200000000,name='UNC_CHA_TOR_INSERTS.IA_MISS.0x40432.14'/,uncore_cha_14/event=0x36,umask=0x21,config1=0x4043200000000,name='UNC_CHA_TOR_OCCUPANCY.IA_MISS.0x40432.14'/}','{uncore_cha_15/event=0x35,umask=0x21,config1=0x4043200000000,name='UNC_CHA_TOR_INSERTS.IA_MISS.0x40432.15'/,uncore_cha_15/event=0x36,umask=0x21,config1=0x4043200000000,name='UNC_CHA_TOR_OCCUPANCY.IA_MISS.0x40432.15'/}','{uncore_cha_16/event=0x35,umask=0x21,config1=0x4043200000000,name='UNC_CHA_TOR_INSERTS.IA_MISS.0x40432.16'/,uncore_cha_16/event=0x36,umask=0x21,config1=0x4043200000000,name='UNC_CHA_TOR_OCCUPANCY.IA_MISS.0x40432.16'/}','{uncore_cha_17/event=0x35,umask=0x21,config1=0x4043200000000,name='UNC_CHA_TOR_INSERTS.IA_MISS.0x40432.17'/,uncore_cha_17/event=0x36,umask=0x21,config1=0x4043200000000,name='UNC_CHA_TOR_OCCUPANCY.IA_MISS.0x40432.17'/}','{uncore_cha_18/event=0x35,umask=0x21,config1=0x4043200000000,name='UNC_CHA_TOR_INSERTS.IA_MISS.0x40432.18'/,uncore_cha_18/event=0x36,umask=0x21,config1=0x4043200000000,name='UNC_CHA_TOR_OCCUPANCY.IA_MISS.0x40432.18'/}','{uncore_cha_19/event=0x35,umask=0x21,config1=0x4043200000000,name='UNC_CHA_TOR_INSERTS.IA_MISS.0x40432.19'/,uncore_cha_19/event=0x36,umask=0x21,config1=0x4043200000000,name='UNC_CHA_TOR_OCCUPANCY.IA_MISS.0x40432.19'/}','{uncore_cha_20/event=0x35,umask=0x21,config1=0x4043200000000,name='UNC_CHA_TOR_INSERTS.IA_MISS.0x40432.20'/,uncore_cha_20/event=0x36,umask=0x21,config1=0x4043200000000,name='UNC_CHA_TOR_OCCUPANCY.IA_MISS.0x40432.20'/}','{uncore_cha_21/event=0x35,umask=0x21,config1=0x4043200000000,name='UNC_CHA_TOR_INSERTS.IA_MISS.0x40432.21'/,uncore_cha_21/event=0x36,umask=0x21,config1=0x4043200000000,name='UNC_CHA_TOR_OCCUPANCY.IA_MISS.0x40432.21'/}','{uncore_cha_22/event=0x35,umask=0x21,config1=0x4043200000000,name='UNC_CHA_TOR_INSERTS.IA_MISS.0x40432.22'/,uncore_cha_22/event=0x36,umask=0x21,config1=0x4043200000000,name='UNC_CHA_TOR_OCCUPANCY.IA_MISS.0x40432.22'/}','{uncore_cha_23/event=0x35,umask=0x21,config1=0x4043200000000,name='UNC_CHA_TOR_INSERTS.IA_MISS.0x40432.23'/,uncore_cha_23/event=0x36,umask=0x21,config1=0x4043200000000,name='UNC_CHA_TOR_OCCUPANCY.IA_MISS.0x40432.23'/}','{uncore_cha_24/event=0x35,umask=0x21,config1=0x4043200000000,name='UNC_CHA_TOR_INSERTS.IA_MISS.0x40432.24'/,uncore_cha_24/event=0x36,umask=0x21,config1=0x4043200000000,name='UNC_CHA_TOR_OCCUPANCY.IA_MISS.0x40432.24'/}','{uncore_cha_25/event=0x35,umask=0x21,config1=0x4043200000000,name='UNC_CHA_TOR_INSERTS.IA_MISS.0x40432.25'/,uncore_cha_25/event=0x36,umask=0x21,config1=0x4043200000000,name='UNC_CHA_TOR_OCCUPANCY.IA_MISS.0x40432.25'/}','{uncore_cha_0/event=0x35,umask=0x21,config1=0x40[431](https://github.com/intel-innersource/applications.analyzers.cloudcompute.perfspect/actions/runs/5159695458/jobs/9294968350#step:6:432)00000000,name='UNC_CHA_TOR_INSERTS.IA_MISS.0x40431.0'/,uncore_cha_0/event=0x36,umask=0x21,config1=0x4043100000000,name='UNC_CHA_TOR_OCCUPANCY.IA_MISS.0x40431.0'/}','{uncore_cha_1/event=0x35,umask=0x21,config1=0x4043100000000,name='UNC_CHA_TOR_INSERTS.IA_MISS.0x40431.1'/,uncore_cha_1/event=0x36,umask=0x21,config1=0x4043100000000,name='UNC_CHA_TOR_OCCUPANCY.IA_MISS.0x40431.1'/}','{uncore_cha_2/event=0x35,umask=0x21,config1=0x4043100000000,name='UNC_CHA_TOR_INSERTS.IA_MISS.0x40431.2'/,uncore_cha_2/event=0x36,umask=0x21,config1=0x4043100000000,name='UNC_CHA_TOR_OCCUPANCY.IA_MISS.0x40431.2'/}','{uncore_cha_3/event=0x35,umask=0x21,config1=0x4043100000000,name='UNC_CHA_TOR_INSERTS.IA_MISS.0x40431.3'/,uncore_cha_3/event=0x36,umask=0x21,config1=0x4043100000000,name='UNC_CHA_TOR_OCCUPANCY.IA_MISS.0x40431.3'/}','{uncore_cha_4/event=0x35,umask=0x21,config1=0x4043100000000,name='UNC_CHA_TOR_INSERTS.IA_MISS.0x40431.4'/,uncore_cha_4/event=0x36,umask=0x21,config1=0x4043100000000,name='UNC_CHA_TOR_OCCUPANCY.IA_MISS.0x40431.4'/}','{uncore_cha_5/event=0x35,umask=0x21,config1=0x4043100000000,name='UNC_CHA_TOR_INSERTS.IA_MISS.0x40431.5'/,uncore_cha_5/event=0x36,umask=0x21,config1=0x4043100000000,name='UNC_CHA_TOR_OCCUPANCY.IA_MISS.0x40431.5'/}','{uncore_cha_6/event=0x35,umask=0x21,config1=0x4043100000000,name='UNC_CHA_TOR_INSERTS.IA_MISS.0x40431.6'/,uncore_cha_6/event=0x36,umask=0x21,config1=0x4043100000000,name='UNC_CHA_TOR_OCCUPANCY.IA_MISS.0x40431.6'/}','{uncore_cha_7/event=0x35,umask=0x21,config1=0x4043100000000,name='UNC_CHA_TOR_INSERTS.IA_MISS.0x40431.7'/,uncore_cha_7/event=0x36,umask=0x21,config1=0x4043100000000,name='UNC_CHA_TOR_OCCUPANCY.IA_MISS.0x40431.7'/}','{uncore_cha_8/event=0x35,umask=0x21,config1=0x4043100000000,name='UNC_CHA_TOR_INSERTS.IA_MISS.0x40431.8'/,uncore_cha_8/event=0x36,umask=0x21,config1=0x4043100000000,name='UNC_CHA_TOR_OCCUPANCY.IA_MISS.0x40431.8'/}','{uncore_cha_9/event=0x35,umask=0x21,config1=0x4043100000000,name='UNC_CHA_TOR_INSERTS.IA_MISS.0x40431.9'/,uncore_cha_9/event=0x36,umask=0x21,config1=0x4043100000000,name='UNC_CHA_TOR_OCCUPANCY.IA_MISS.0x40431.9'/}','{uncore_cha_10/event=0x35,umask=0x21,config1=0x4043100000000,name='UNC_CHA_TOR_INSERTS.IA_MISS.0x40431.10'/,uncore_cha_10/event=0x36,umask=0x21,config1=0x4043100000000,name='UNC_CHA_TOR_OCCUPANCY.IA_MISS.0x40431.10'/}','{uncore_cha_11/event=0x35,umask=0x21,config1=0x4043100000000,name='UNC_CHA_TOR_INSERTS.IA_MISS.0x40431.11'/,uncore_cha_11/event=0x36,umask=0x21,config1=0x4043100000000,name='UNC_CHA_TOR_OCCUPANCY.IA_MISS.0x40431.11'/}','{uncore_cha_12/event=0x35,umask=0x21,config1=0x4043100000000,name='UNC_CHA_TOR_INSERTS.IA_MISS.0x40431.12'/,uncore_cha_12/event=0x36,umask=0x21,config1=0x4043100000000,name='UNC_CHA_TOR_OCCUPANCY.IA_MISS.0x40431.12'/}','{uncore_cha_13/event=0x35,umask=0x21,config1=0x4043100000000,name='UNC_CHA_TOR_INSERTS.IA_MISS.0x40431.13'/,uncore_cha_13/event=0x36,umask=0x21,config1=0x4043100000000,name='UNC_CHA_TOR_OCCUPANCY.IA_MISS.0x40431.13'/}','{uncore_cha_14/event=0x35,umask=0x21,config1=0x4043100000000,name='UNC_CHA_TOR_INSERTS.IA_MISS.0x40431.14'/,uncore_cha_14/event=0x36,umask=0x21,config1=0x4043100000000,name='UNC_CHA_TOR_OCCUPANCY.IA_MISS.0x40431.14'/}','{uncore_cha_15/event=0x35,umask=0x21,config1=0x4043100000000,name='UNC_CHA_TOR_INSERTS.IA_MISS.0x40431.15'/,uncore_cha_15/event=0x36,umask=0x21,config1=0x4043100000000,name='UNC_CHA_TOR_OCCUPANCY.IA_MISS.0x40431.15'/}','{uncore_cha_16/event=0x35,umask=0x21,config1=0x4043100000000,name='UNC_CHA_TOR_INSERTS.IA_MISS.0x40431.16'/,uncore_cha_16/event=0x36,umask=0x21,config1=0x4043100000000,name='UNC_CHA_TOR_OCCUPANCY.IA_MISS.0x40431.16'/}','{uncore_cha_17/event=0x35,umask=0x21,config1=0x4043100000000,name='UNC_CHA_TOR_INSERTS.IA_MISS.0x40431.17'/,uncore_cha_17/event=0x36,umask=0x21,config1=0x4043100000000,name='UNC_CHA_TOR_OCCUPANCY.IA_MISS.0x40431.17'/}','{uncore_cha_18/event=0x35,umask=0x21,config1=0x4043100000000,name='UNC_CHA_TOR_INSERTS.IA_MISS.0x40431.18'/,uncore_cha_18/event=0x36,umask=0x21,config1=0x4043100000000,name='UNC_CHA_TOR_OCCUPANCY.IA_MISS.0x40431.18'/}','{uncore_cha_19/event=0x35,umask=0x21,config1=0x4043100000000,name='UNC_CHA_TOR_INSERTS.IA_MISS.0x40431.19'/,uncore_cha_19/event=0x36,umask=0x21,config1=0x4043100000000,name='UNC_CHA_TOR_OCCUPANCY.IA_MISS.0x40431.19'/}','{uncore_cha_20/event=0x35,umask=0x21,config1=0x4043100000000,name='UNC_CHA_TOR_INSERTS.IA_MISS.0x40431.20'/,uncore_cha_20/event=0x36,umask=0x21,config1=0x4043100000000,name='UNC_CHA_TOR_OCCUPANCY.IA_MISS.0x40431.20'/}','{uncore_cha_21/event=0x35,umask=0x21,config1=0x4043100000000,name='UNC_CHA_TOR_INSERTS.IA_MISS.0x40431.21'/,uncore_cha_21/event=0x36,umask=0x21,config1=0x4043100000000,name='UNC_CHA_TOR_OCCUPANCY.IA_MISS.0x40431.21'/}','{uncore_cha_22/event=0x35,umask=0x21,config1=0x4043100000000,name='UNC_CHA_TOR_INSERTS.IA_MISS.0x40431.22'/,uncore_cha_22/event=0x36,umask=0x21,config1=0x4043100000000,name='UNC_CHA_TOR_OCCUPANCY.IA_MISS.0x40431.22'/}','{uncore_cha_23/event=0x35,umask=0x21,config1=0x4043100000000,name='UNC_CHA_TOR_INSERTS.IA_MISS.0x40431.23'/,uncore_cha_23/event=0x36,umask=0x21,config1=0x4043100000000,name='UNC_CHA_TOR_OCCUPANCY.IA_MISS.0x40431.23'/}','{uncore_cha_24/event=0x35,umask=0x21,config1=0x4043100000000,name='UNC_CHA_TOR_INSERTS.IA_MISS.0x40431.24'/,uncore_cha_24/event=0x36,umask=0x21,config1=0x4043100000000,name='UNC_CHA_TOR_OCCUPANCY.IA_MISS.0x40431.24'/}','{uncore_cha_25/event=0x35,umask=0x21,config1=0x4043100000000,name='UNC_CHA_TOR_INSERTS.IA_MISS.0x40431.25'/,uncore_cha_25/event=0x36,umask=0x21,config1=0x4043100000000,name='UNC_CHA_TOR_OCCUPANCY.IA_MISS.0x40431.25'/}','{uncore_cha_0/event=0x35,umask=0x21,config1=0x12CC023300000000,name='UNC_CHA_TOR_INSERTS.IA_MISS.0x12CC0233.0'/}','{uncore_cha_1/event=0x35,umask=0x21,config1=0x12CC023300000000,name='UNC_CHA_TOR_INSERTS.IA_MISS.0x12CC0233.1'/}','{uncore_cha_2/event=0x35,umask=0x21,config1=0x12CC023300000000,name='UNC_CHA_TOR_INSERTS.IA_MISS.0x12CC0233.2'/}','{uncore_cha_3/event=0x35,umask=0x21,config1=0x12CC023300000000,name='UNC_CHA_TOR_INSERTS.IA_MISS.0x12CC0233.3'/}','{uncore_cha_4/event=0x35,umask=0x21,config1=0x12CC023300000000,name='UNC_CHA_TOR_INSERTS.IA_MISS.0x12CC0233.4'/}','{uncore_cha_5/event=0x35,umask=0x21,config1=0x12CC023300000000,name='UNC_CHA_TOR_INSERTS.IA_MISS.0x12CC0233.5'/}','{uncore_cha_6/event=0x35,umask=0x21,config1=0x12CC023300000000,name='UNC_CHA_TOR_INSERTS.IA_MISS.0x12CC0233.6'/}','{uncore_cha_7/event=0x35,umask=0x21,config1=0x12CC023300000000,name='UNC_CHA_TOR_INSERTS.IA_MISS.0x12CC0233.7'/}','{uncore_cha_8/event=0x35,umask=0x21,config1=0x12CC023300000000,name='UNC_CHA_TOR_INSERTS.IA_MISS.0x12CC0233.8'/}','{uncore_cha_9/event=0x35,umask=0x21,config1=0x12CC023300000000,name='UNC_CHA_TOR_INSERTS.IA_MISS.0x12CC0233.9'/}','{uncore_cha_10/event=0x35,umask=0x21,config1=0x12CC023300000000,name='UNC_CHA_TOR_INSERTS.IA_MISS.0x12CC0233.10'/}','{uncore_cha_11/event=0x35,umask=0x21,config1=0x12CC023300000000,name='UNC_CHA_TOR_INSERTS.IA_MISS.0x12CC0233.11'/}','{uncore_cha_12/event=0x35,umask=0x21,config1=0x12CC023300000000,name='UNC_CHA_TOR_INSERTS.IA_MISS.0x12CC0233.12'/}','{uncore_cha_13/event=0x35,umask=0x21,config1=0x12CC023300000000,name='UNC_CHA_TOR_INSERTS.IA_MISS.0x12CC0233.13'/}','{uncore_cha_14/event=0x35,umask=0x21,config1=0x12CC023300000000,name='UNC_CHA_TOR_INSERTS.IA_MISS.0x12CC0233.14'/}','{uncore_cha_15/event=0x35,umask=0x21,config1=0x12CC023300000000,name='UNC_CHA_TOR_INSERTS.IA_MISS.0x12CC0233.15'/}','{uncore_cha_16/event=0x35,umask=0x21,config1=0x12CC023300000000,name='UNC_CHA_TOR_INSERTS.IA_MISS.0x12CC0233.16'/}','{uncore_cha_17/event=0x35,umask=0x21,config1=0x12CC023300000000,name='UNC_CHA_TOR_INSERTS.IA_MISS.0x12CC0233.17'/}','{uncore_cha_18/event=0x35,umask=0x21,config1=0x12CC023300000000,name='UNC_CHA_TOR_INSERTS.IA_MISS.0x12CC0233.18'/}','{uncore_cha_19/event=0x35,umask=0x21,config1=0x12CC023300000000,name='UNC_CHA_TOR_INSERTS.IA_MISS.0x12CC0233.19'/}','{uncore_cha_20/event=0x35,umask=0x21,config1=0x12CC023300000000,name='UNC_CHA_TOR_INSERTS.IA_MISS.0x12CC0233.20'/}','{uncore_cha_21/event=0x35,umask=0x21,config1=0x12CC023300000000,name='UNC_CHA_TOR_INSERTS.IA_MISS.0x12CC0233.21'/}','{uncore_cha_22/event=0x35,umask=0x21,config1=0x12CC023300000000,name='UNC_CHA_TOR_INSERTS.IA_MISS.0x12CC0233.22'/}','{uncore_cha_23/event=0x35,umask=0x21,config1=0x12CC023300000000,name='UNC_CHA_TOR_INSERTS.IA_MISS.0x12CC0233.23'/}','{uncore_cha_24/event=0x35,umask=0x21,config1=0x12CC023300000000,name='UNC_CHA_TOR_INSERTS.IA_MISS.0x12CC0233.24'/}','{uncore_cha_25/event=0x35,umask=0x21,config1=0x12CC023300000000,name='UNC_CHA_TOR_INSERTS.IA_MISS.0x12CC0233.25'/}','{uncore_cha_0/event=0x35,umask=0x21,config1=0x12D4043300000000,name='UNC_CHA_TOR_INSERTS.IA_MISS.0x12D40433.0'/}','{uncore_cha_1/event=0x35,umask=0x21,config1=0x12D4043300000000,name='UNC_CHA_TOR_INSERTS.IA_MISS.0x12D40433.1'/}','{uncore_cha_2/event=0x35,umask=0x21,config1=0x12D4043300000000,name='UNC_CHA_TOR_INSERTS.IA_MISS.0x12D40433.2'/}','{uncore_cha_3/event=0x35,umask=0x21,config1=0x12D4043300000000,name='UNC_CHA_TOR_INSERTS.IA_MISS.0x12D40433.3'/}','{uncore_cha_4/event=0x35,umask=0x21,config1=0x12D4043300000000,name='UNC_CHA_TOR_INSERTS.IA_MISS.0x12D40433.4'/}','{uncore_cha_5/event=0x35,umask=0x21,config1=0x12D4043300000000,name='UNC_CHA_TOR_INSERTS.IA_MISS.0x12D40433.5'/}','{uncore_cha_6/event=0x35,umask=0x21,config1=0x12D4043300000000,name='UNC_CHA_TOR_INSERTS.IA_MISS.0x12D40433.6'/}','{uncore_cha_7/event=0x35,umask=0x21,config1=0x12D4043300000000,name='UNC_CHA_TOR_INSERTS.IA_MISS.0x12D40433.7'/}','{uncore_cha_8/event=0x35,umask=0x21,config1=0x12D4043300000000,name='UNC_CHA_TOR_INSERTS.IA_MISS.0x12D40433.8'/}','{uncore_cha_9/event=0x35,umask=0x21,config1=0x12D4043300000000,name='UNC_CHA_TOR_INSERTS.IA_MISS.0x12D40433.9'/}','{uncore_cha_10/event=0x35,umask=0x21,config1=0x12D4043300000000,name='UNC_CHA_TOR_INSERTS.IA_MISS.0x12D40433.10'/}','{uncore_cha_11/event=0x35,umask=0x21,config1=0x12D4043300000000,name='UNC_CHA_TOR_INSERTS.IA_MISS.0x12D40433.11'/}','{uncore_cha_12/event=0x35,umask=0x21,config1=0x12D4043300000000,name='UNC_CHA_TOR_INSERTS.IA_MISS.0x12D40433.12'/}','{uncore_cha_13/event=0x35,umask=0x21,config1=0x12D4043300000000,name='UNC_CHA_TOR_INSERTS.IA_MISS.0x12D40433.13'/}','{uncore_cha_14/event=0x35,umask=0x21,config1=0x12D4043300000000,name='UNC_CHA_TOR_INSERTS.IA_MISS.0x12D40433.14'/}','{uncore_cha_15/event=0x35,umask=0x21,config1=0x12D4043300000000,name='UNC_CHA_TOR_INSERTS.IA_MISS.0x12D40433.15'/}','{uncore_cha_16/event=0x35,umask=0x21,config1=0x12D4043300000000,name='UNC_CHA_TOR_INSERTS.IA_MISS.0x12D40433.16'/}','{uncore_cha_17/event=0x35,umask=0x21,config1=0x12D4043300000000,name='UNC_CHA_TOR_INSERTS.IA_MISS.0x12D40433.17'/}','{uncore_cha_18/event=0x35,umask=0x21,config1=0x12D4043300000000,name='UNC_CHA_TOR_INSERTS.IA_MISS.0x12D40433.18'/}','{uncore_cha_19/event=0x35,umask=0x21,config1=0x12D4043300000000,name='UNC_CHA_TOR_INSERTS.IA_MISS.0x12D40433.19'/}','{uncore_cha_20/event=0x35,umask=0x21,config1=0x12D4043300000000,name='UNC_CHA_TOR_INSERTS.IA_MISS.0x12D40433.20'/}','{uncore_cha_21/event=0x35,umask=0x21,config1=0x12D4043300000000,name='UNC_CHA_TOR_INSERTS.IA_MISS.0x12D40433.21'/}','{uncore_cha_22/event=0x35,umask=0x21,config1=0x12D4043300000000,name='UNC_CHA_TOR_INSERTS.IA_MISS.0x12D40433.22'/}','{uncore_cha_23/event=0x35,umask=0x21,config1=0x12D4043300000000,name='UNC_CHA_TOR_INSERTS.IA_MISS.0x12D40433.23'/}','{uncore_cha_24/event=0x35,umask=0x21,config1=0x12D4043300000000,name='UNC_CHA_TOR_INSERTS.IA_MISS.0x12D40433.24'/}','{uncore_cha_25/event=0x35,umask=0x21,config1=0x12D4043300000000,name='UNC_CHA_TOR_INSERTS.IA_MISS.0x12D40433.25'/}','{uncore_cha_0/event=0x35,umask=0x21,config1=0x12C4003300000000,name='UNC_CHA_TOR_INSERTS.IA_MISS.0x12C40033.0'/}','{uncore_cha_1/event=0x35,umask=0x21,config1=0x12C4003300000000,name='UNC_CHA_TOR_INSERTS.IA_MISS.0x12C40033.1'/}','{uncore_cha_2/event=0x35,umask=0x21,config1=0x12C4003300000000,name='UNC_CHA_TOR_INSERTS.IA_MISS.0x12C40033.2'/}','{uncore_cha_3/event=0x35,umask=0x21,config1=0x12C4003300000000,name='UNC_CHA_TOR_INSERTS.IA_MISS.0x12C40033.3'/}','{uncore_cha_4/event=0x35,umask=0x21,config1=0x12C4003300000000,name='UNC_CHA_TOR_INSERTS.IA_MISS.0x12C40033.4'/}','{uncore_cha_5/event=0x35,umask=0x21,config1=0x12C4003300000000,name='UNC_CHA_TOR_INSERTS.IA_MISS.0x12C40033.5'/}','{uncore_cha_6/event=0x35,umask=0x21,config1=0x12C4003300000000,name='UNC_CHA_TOR_INSERTS.IA_MISS.0x12C40033.6'/}','{uncore_cha_7/event=0x35,umask=0x21,config1=0x12C4003300000000,name='UNC_CHA_TOR_INSERTS.IA_MISS.0x12C40033.7'/}','{uncore_cha_8/event=0x35,umask=0x21,config1=0x12C4003300000000,name='UNC_CHA_TOR_INSERTS.IA_MISS.0x12C40033.8'/}','{uncore_cha_9/event=0x35,umask=0x21,config1=0x12C4003300000000,name='UNC_CHA_TOR_INSERTS.IA_MISS.0x12C40033.9'/}','{uncore_cha_10/event=0x35,umask=0x21,config1=0x12C4003300000000,name='UNC_CHA_TOR_INSERTS.IA_MISS.0x12C40033.10'/}','{uncore_cha_11/event=0x35,umask=0x21,config1=0x12C4003300000000,name='UNC_CHA_TOR_INSERTS.IA_MISS.0x12C40033.11'/}','{uncore_cha_12/event=0x35,umask=0x21,config1=0x12C4003300000000,name='UNC_CHA_TOR_INSERTS.IA_MISS.0x12C40033.12'/}','{uncore_cha_13/event=0x35,umask=0x21,config1=0x12C4003300000000,name='UNC_CHA_TOR_INSERTS.IA_MISS.0x12C40033.13'/}','{uncore_cha_14/event=0x35,umask=0x21,config1=0x12C4003300000000,name='UNC_CHA_TOR_INSERTS.IA_MISS.0x12C40033.14'/}','{uncore_cha_15/event=0x35,umask=0x21,config1=0x12C4003300000000,name='UNC_CHA_TOR_INSERTS.IA_MISS.0x12C40033.15'/}','{uncore_cha_16/event=0x35,umask=0x21,config1=0x12C4003300000000,name='UNC_CHA_TOR_INSERTS.IA_MISS.0x12C40033.16'/}','{uncore_cha_17/event=0x35,umask=0x21,config1=0x12C4003300000000,name='UNC_CHA_TOR_INSERTS.IA_MISS.0x12C40033.17'/}','{uncore_cha_18/event=0x35,umask=0x21,config1=0x12C4003300000000,name='UNC_CHA_TOR_INSERTS.IA_MISS.0x12C40033.18'/}','{uncore_cha_19/event=0x35,umask=0x21,config1=0x12C4003300000000,name='UNC_CHA_TOR_INSERTS.IA_MISS.0x12C40033.19'/}','{uncore_cha_20/event=0x35,umask=0x21,config1=0x12C4003300000000,name='UNC_CHA_TOR_INSERTS.IA_MISS.0x12C40033.20'/}','{uncore_cha_21/event=0x35,umask=0x21,config1=0x12C4003300000000,name='UNC_CHA_TOR_INSERTS.IA_MISS.0x12C40033.21'/}','{uncore_cha_22/event=0x35,umask=0x21,config1=0x12C4003300000000,name='UNC_CHA_TOR_INSERTS.IA_MISS.0x12C40033.22'/}','{uncore_cha_23/event=0x35,umask=0x21,config1=0x12C4003300000000,name='UNC_CHA_TOR_INSERTS.IA_MISS.0x12C40033.23'/}','{uncore_cha_24/event=0x35,umask=0x21,config1=0x12C4003300000000,name='UNC_CHA_TOR_INSERTS.IA_MISS.0x12C40033.24'/}','{uncore_cha_25/event=0x35,umask=0x21,config1=0x12C4003300000000,name='UNC_CHA_TOR_INSERTS.IA_MISS.0x12C40033.25'/}','{uncore_iio_0/event=0x83,umask=0x04,ch_mask=0x00,fc_mask=0x07,name='UNC_IIO_DATA_REQ_OF_CPU.MEM_READ.PART0.0'/,uncore_iio_0/event=0x83,umask=0x04,ch_mask=0x02,fc_mask=0x07,name='UNC_IIO_DATA_REQ_OF_CPU.MEM_READ.PART1.0'/}','{uncore_iio_1/event=0x83,umask=0x04,ch_mask=0x00,fc_mask=0x07,name='UNC_IIO_DATA_REQ_OF_CPU.MEM_READ.PART0.1'/,uncore_iio_1/event=0x83,umask=0x04,ch_mask=0x02,fc_mask=0x07,name='UNC_IIO_DATA_REQ_OF_CPU.MEM_READ.PART1.1'/}','{uncore_iio_2/event=0x83,umask=0x04,ch_mask=0x00,fc_mask=0x07,name='UNC_IIO_DATA_REQ_OF_CPU.MEM_READ.PART0.2'/,uncore_iio_2/event=0x83,umask=0x04,ch_mask=0x02,fc_mask=0x07,name='UNC_IIO_DATA_REQ_OF_CPU.MEM_READ.PART1.2'/}','{uncore_iio_3/event=0x83,umask=0x04,ch_mask=0x00,fc_mask=0x07,name='UNC_IIO_DATA_REQ_OF_CPU.MEM_READ.PART0.3'/,uncore_iio_3/event=0x83,umask=0x04,ch_mask=0x02,fc_mask=0x07,name='UNC_IIO_DATA_REQ_OF_CPU.MEM_READ.PART1.3'/}','{uncore_iio_4/event=0x83,umask=0x04,ch_mask=0x00,fc_mask=0x07,name='UNC_IIO_DATA_REQ_OF_CPU.MEM_READ.PART0.4'/,uncore_iio_4/event=0x83,umask=0x04,ch_mask=0x02,fc_mask=0x07,name='UNC_IIO_DATA_REQ_OF_CPU.MEM_READ.PART1.4'/}','{uncore_iio_5/event=0x83,umask=0x04,ch_mask=0x00,fc_mask=0x07,name='UNC_IIO_DATA_REQ_OF_CPU.MEM_READ.PART0.5'/,uncore_iio_5/event=0x83,umask=0x04,ch_mask=0x02,fc_mask=0x07,name='UNC_IIO_DATA_REQ_OF_CPU.MEM_READ.PART1.5'/}','{uncore_iio_0/event=0x83,umask=0x04,ch_mask=0x04,fc_mask=0x07,name='UNC_IIO_DATA_REQ_OF_CPU.MEM_READ.PART2.0'/,uncore_iio_0/event=0x83,umask=0x04,ch_mask=0x08,fc_mask=0x07,name='UNC_IIO_DATA_REQ_OF_CPU.MEM_READ.PART3.0'/}','{uncore_iio_1/event=0x83,umask=0x04,ch_mask=0x04,fc_mask=0x07,name='UNC_IIO_DATA_REQ_OF_CPU.MEM_READ.PART2.1'/,uncore_iio_1/event=0x83,umask=0x04,ch_mask=0x08,fc_mask=0x07,name='UNC_IIO_DATA_REQ_OF_CPU.MEM_READ.PART3.1'/}','{uncore_iio_2/event=0x83,umask=0x04,ch_mask=0x04,fc_mask=0x07,name='UNC_IIO_DATA_REQ_OF_CPU.MEM_READ.PART2.2'/,uncore_iio_2/event=0x83,umask=0x04,ch_mask=0x08,fc_mask=0x07,name='UNC_IIO_DATA_REQ_OF_CPU.MEM_READ.PART3.2'/}','{uncore_iio_3/event=0x83,umask=0x04,ch_mask=0x04,fc_mask=0x07,name='UNC_IIO_DATA_REQ_OF_CPU.MEM_READ.PART2.3'/,uncore_iio_3/event=0x83,umask=0x04,ch_mask=0x08,fc_mask=0x07,name='UNC_IIO_DATA_REQ_OF_CPU.MEM_READ.PART3.3'/}','{uncore_iio_4/event=0x83,umask=0x04,ch_mask=0x04,fc_mask=0x07,name='UNC_IIO_DATA_REQ_OF_CPU.MEM_READ.PART2.4'/,uncore_iio_4/event=0x83,umask=0x04,ch_mask=0x08,fc_mask=0x07,name='UNC_IIO_DATA_REQ_OF_CPU.MEM_READ.PART3.4'/}','{uncore_iio_5/event=0x83,umask=0x04,ch_mask=0x04,fc_mask=0x07,name='UNC_IIO_DATA_REQ_OF_CPU.MEM_READ.PART2.5'/,uncore_iio_5/event=0x83,umask=0x04,ch_mask=0x08,fc_mask=0x07,name='UNC_IIO_DATA_REQ_OF_CPU.MEM_READ.PART3.5'/}','{uncore_iio_0/event=0x83,umask=0x01,ch_mask=0x00,fc_mask=0x07,name='UNC_IIO_DATA_REQ_OF_CPU.MEM_WRITE.PART0.0'/,uncore_iio_0/event=0x83,umask=0x01,ch_mask=0x02,fc_mask=0x07,name='UNC_IIO_DATA_REQ_OF_CPU.MEM_WRITE.PART1.0'/}','{uncore_iio_1/event=0x83,umask=0x01,ch_mask=0x00,fc_mask=0x07,name='UNC_IIO_DATA_REQ_OF_CPU.MEM_WRITE.PART0.1'/,uncore_iio_1/event=0x83,umask=0x01,ch_mask=0x02,fc_mask=0x07,name='UNC_IIO_DATA_REQ_OF_CPU.MEM_WRITE.PART1.1'/}','{uncore_iio_2/event=0x83,umask=0x01,ch_mask=0x00,fc_mask=0x07,name='UNC_IIO_DATA_REQ_OF_CPU.MEM_WRITE.PART0.2'/,uncore_iio_2/event=0x83,umask=0x01,ch_mask=0x02,fc_mask=0x07,name='UNC_IIO_DATA_REQ_OF_CPU.MEM_WRITE.PART1.2'/}','{uncore_iio_3/event=0x83,umask=0x01,ch_mask=0x00,fc_mask=0x07,name='UNC_IIO_DATA_REQ_OF_CPU.MEM_WRITE.PART0.3'/,uncore_iio_3/event=0x83,umask=0x01,ch_mask=0x02,fc_mask=0x07,name='UNC_IIO_DATA_REQ_OF_CPU.MEM_WRITE.PART1.3'/}','{uncore_iio_4/event=0x83,umask=0x01,ch_mask=0x00,fc_mask=0x07,name='UNC_IIO_DATA_REQ_OF_CPU.MEM_WRITE.PART0.4'/,uncore_iio_4/event=0x83,umask=0x01,ch_mask=0x02,fc_mask=0x07,name='UNC_IIO_DATA_REQ_OF_CPU.MEM_WRITE.PART1.4'/}','{uncore_iio_5/event=0x83,umask=0x01,ch_mask=0x00,fc_mask=0x07,name='UNC_IIO_DATA_REQ_OF_CPU.MEM_WRITE.PART0.5'/,uncore_iio_5/event=0x83,umask=0x01,ch_mask=0x02,fc_mask=0x07,name='UNC_IIO_DATA_REQ_OF_CPU.MEM_WRITE.PART1.5'/}','{uncore_iio_0/event=0x83,umask=0x01,ch_mask=0x04,fc_mask=0x07,name='UNC_IIO_DATA_REQ_OF_CPU.MEM_WRITE.PART2.0'/,uncore_iio_0/event=0x83,umask=0x01,ch_mask=0x08,fc_mask=0x07,name='UNC_IIO_DATA_REQ_OF_CPU.MEM_WRITE.PART3.0'/}','{uncore_iio_1/event=0x83,umask=0x01,ch_mask=0x04,fc_mask=0x07,name='UNC_IIO_DATA_REQ_OF_CPU.MEM_WRITE.PART2.1'/,uncore_iio_1/event=0x83,umask=0x01,ch_mask=0x08,fc_mask=0x07,name='UNC_IIO_DATA_REQ_OF_CPU.MEM_WRITE.PART3.1'/}','{uncore_iio_2/event=0x83,umask=0x01,ch_mask=0x04,fc_mask=0x07,name='UNC_IIO_DATA_REQ_OF_CPU.MEM_WRITE.PART2.2'/,uncore_iio_2/event=0x83,umask=0x01,ch_mask=0x08,fc_mask=0x07,name='UNC_IIO_DATA_REQ_OF_CPU.MEM_WRITE.PART3.2'/}','{uncore_iio_3/event=0x83,umask=0x01,ch_mask=0x04,fc_mask=0x07,name='UNC_IIO_DATA_REQ_OF_CPU.MEM_WRITE.PART2.3'/,uncore_iio_3/event=0x83,umask=0x01,ch_mask=0x08,fc_mask=0x07,name='UNC_IIO_DATA_REQ_OF_CPU.MEM_WRITE.PART3.3'/}','{uncore_iio_4/event=0x83,umask=0x01,ch_mask=0x04,fc_mask=0x07,name='UNC_IIO_DATA_REQ_OF_CPU.MEM_WRITE.PART2.4'/,uncore_iio_4/event=0x83,umask=0x01,ch_mask=0x08,fc_mask=0x07,name='UNC_IIO_DATA_REQ_OF_CPU.MEM_WRITE.PART3.4'/}','{uncore_iio_5/event=0x83,umask=0x01,ch_mask=0x04,fc_mask=0x07,name='UNC_IIO_DATA_REQ_OF_CPU.MEM_WRITE.PART2.5'/,uncore_iio_5/event=0x83,umask=0x01,ch_mask=0x08,fc_mask=0x07,name='UNC_IIO_DATA_REQ_OF_CPU.MEM_WRITE.PART3.5'/}','{uncore_upi_0/event=0x2,umask=0x0f,name='UNC_UPI_TxL_FLITS.ALL_DATA.0'/,uncore_upi_0/event=0x2,umask=0x97,name='UNC_UPI_TxL_FLITS.NON_DATA.0'/,uncore_upi_0/event=0x1,umask=0x0,name='UNC_UPI_CLOCKTICKS.0'/,uncore_upi_0/event=0x21,umask=0x0,name='UNC_UPI_L1_POWER_CYCLES.0'/}','{uncore_upi_1/event=0x2,umask=0x0f,name='UNC_UPI_TxL_FLITS.ALL_DATA.1'/,uncore_upi_1/event=0x2,umask=0x97,name='UNC_UPI_TxL_FLITS.NON_DATA.1'/,uncore_upi_1/event=0x1,umask=0x0,name='UNC_UPI_CLOCKTICKS.1'/,uncore_upi_1/event=0x21,umask=0x0,name='UNC_UPI_L1_POWER_CYCLES.1'/}','{uncore_upi_2/event=0x2,umask=0x0f,name='UNC_UPI_TxL_FLITS.ALL_DATA.2'/,uncore_upi_2/event=0x2,umask=0x97,name='UNC_UPI_TxL_FLITS.NON_DATA.2'/,uncore_upi_2/event=0x1,umask=0x0,name='UNC_UPI_CLOCKTICKS.2'/,uncore_upi_2/event=0x21,umask=0x0,name='UNC_UPI_L1_POWER_CYCLES.2'/}' sleep 10
2023-06-02 21:04:41,916 INFO: Collecting perf stat for events in : clx_skx.txt
2023-06-02 21:04:57,211 INFO: Collection complete! Calculating TSC frequency now
2023-06-02 21:04:58,249 INFO: nmi_watchdog enabled!
2023-06-02 21:04:58,268 INFO: perf stat dumped to perfstat.csv
2023-06-02 21:04:59,288 WARNING: MULTIPLE GROUPS: metric "metric_LLC MPI (includes code+data+rfo w/ prefetches)", events "{'UNC_CHA_TOR_INSERTS.IA_MISS.0x12C40033', 'instructions', 'UNC_CHA_TOR_INSERTS.IA_MISS.0x12D40433', 'UNC_CHA_TOR_INSERTS.IA_MISS.0x12CC0233'}"
2023-06-02 21:04:59,289 WARNING: MULTIPLE GROUPS: metric "metric_LLC code read MPI (demand+prefetch)", events "{'instructions', 'UNC_CHA_TOR_INSERTS.IA_MISS.0x12CC0233'}"
2023-06-02 21:04:59,289 WARNING: MULTIPLE GROUPS: metric "metric_LLC data read MPI (demand+prefetch)", events "{'instructions', 'UNC_CHA_TOR_INSERTS.IA_MISS.0x12D40433'}"
2023-06-02 21:04:59,290 WARNING: MULTIPLE GROUPS: metric "metric_LLC total HITM (per instr)", events "{'instructions', 'OCR.ALL_READS.L3_MISS.REMOTE_HITM'}"
2023-06-02 21:04:59,290 WARNING: MULTIPLE GROUPS: metric "metric_LLC total HIT clean line forwards (per instr)", events "{'instructions', 'OCR.ALL_READS.L3_MISS.REMOTE_HIT_FORWARD'}"
2023-06-02 21:04:59,290 WARNING: MULTIPLE GROUPS: metric "metric_Average LLC data read miss latency (in clks)", events "{'OFFCORE_REQUESTS.L3_MISS_DEMAND_DATA_RD', 'OFFCORE_REQUESTS_OUTSTANDING.L3_MISS_DEMAND_DATA_RD'}"
2023-06-02 21:04:59,292 WARNING: MULTIPLE GROUPS: metric "metric_Average LLC data read miss latency for LOCAL requests (in ns)", events "{'UNC_CHA_TOR_OCCUPANCY.IA_MISS.0x40432', 'UNC_CHA_CLOCKTICKS', 'UNC_CHA_TOR_INSERTS.IA_MISS.0x40432'}"
2023-06-02 21:04:59,292 WARNING: MULTIPLE GROUPS: metric "metric_Average LLC data read miss latency for REMOTE requests (in ns)", events "{'UNC_CHA_TOR_OCCUPANCY.IA_MISS.0x40431', 'UNC_CHA_CLOCKTICKS', 'UNC_CHA_TOR_INSERTS.IA_MISS.0x40431'}"
2023-06-02 21:04:59,295 WARNING: MULTIPLE GROUPS: metric "metric_NUMA %_Reads addressed to local DRAM", events "{'UNC_CHA_TOR_INSERTS.IA_MISS.0x40431', 'UNC_CHA_TOR_INSERTS.IA_MISS.0x40432'}"
2023-06-02 21:04:59,296 WARNING: MULTIPLE GROUPS: metric "metric_NUMA %_Reads addressed to remote DRAM", events "{'UNC_CHA_TOR_INSERTS.IA_MISS.0x40431', 'UNC_CHA_TOR_INSERTS.IA_MISS.0x40432'}"
2023-06-02 21:04:59,297 WARNING: MULTIPLE GROUPS: metric "metric_UPI Transmit utilization_% (includes control)", events "{'UNC_UPI_CLOCKTICKS', 'UNC_UPI_TxL_FLITS.ALL_DATA', 'cstate_pkg/c6-residency/', 'UNC_UPI_L1_POWER_CYCLES', 'UNC_UPI_TxL_FLITS.NON_DATA'}"
2023-06-02 21:04:59,297 WARNING: MISSING EVENTS: metric "metric_UPI Transmit utilization_% (includes control)" events "['cstate_pkg/c6-residency/']"
2023-06-02 21:04:59,298 WARNING: MISSING EVENTS: metric "metric_core c6 residency %" events "['cstate_core/c6-residency/']"
2023-06-02 21:04:59,298 WARNING: MISSING EVENTS: metric "metric_package c6 residency %" events "['cstate_pkg/c6-residency/']"
2023-06-02 21:04:59,301 WARNING: MULTIPLE GROUPS: metric "metric_IO_bandwidth_disk_or_network_writes (MB/sec)", events "{'UNC_IIO_DATA_REQ_OF_CPU.MEM_READ.PART1', 'UNC_IIO_DATA_REQ_OF_CPU.MEM_READ.PART0', 'UNC_IIO_DATA_REQ_OF_CPU.MEM_READ.PART2', 'UNC_IIO_DATA_REQ_OF_CPU.MEM_READ.PART3'}"
2023-06-02 21:04:59,302 WARNING: MULTIPLE GROUPS: metric "metric_IO_bandwidth_disk_or_network_reads (MB/sec)", events "{'UNC_IIO_DATA_REQ_OF_CPU.MEM_WRITE.PART1', 'UNC_IIO_DATA_REQ_OF_CPU.MEM_WRITE.PART3', 'UNC_IIO_DATA_REQ_OF_CPU.MEM_WRITE.PART0', 'UNC_IIO_DATA_REQ_OF_CPU.MEM_WRITE.PART2'}"
2023-06-02 21:04:59,303 WARNING: MULTIPLE GROUPS: metric "metric_TMA_..Frontend_Latency(%)", events "{'IDQ_UOPS_NOT_DELIVERED.CYCLES_0_UOPS_DELIV.CORE', 'CPU_CLK_UNHALTED.THREAD_ANY'}"
2023-06-02 21:04:59,305 WARNING: MULTIPLE GROUPS: metric "metric_TMA_....Branch_Resteers(%)", events "{'BACLEARS.ANY', 'INT_MISC.CLEAR_RESTEER_CYCLES', 'cpu-cycles'}"
2023-06-02 21:04:59,307 WARNING: MULTIPLE GROUPS: metric "metric_TMA_..Frontend_Bandwidth(%)", events "{'IDQ_UOPS_NOT_DELIVERED.CYCLES_0_UOPS_DELIV.CORE', 'IDQ_UOPS_NOT_DELIVERED.CORE', 'CPU_CLK_UNHALTED.THREAD_ANY'}"
2023-06-02 21:04:59,307 WARNING: MULTIPLE GROUPS: metric "metric_TMA_....MITE(%)", events "{'IDQ.ALL_MITE_CYCLES_ANY_UOPS', 'cpu-cycles', 'IDQ.ALL_MITE_CYCLES_4_UOPS', 'CPU_CLK_UNHALTED.THREAD_ANY'}"
2023-06-02 21:04:59,308 WARNING: MULTIPLE GROUPS: metric "metric_TMA_....DSB(%)", events "{'IDQ.ALL_DSB_CYCLES_ANY_UOPS', 'cpu-cycles', 'IDQ.ALL_DSB_CYCLES_4_UOPS', 'CPU_CLK_UNHALTED.THREAD_ANY'}"
2023-06-02 21:04:59,309 WARNING: MULTIPLE GROUPS: metric "metric_TMA_Bad_Speculation(%)", events "{'UOPS_ISSUED.ANY', 'UOPS_RETIRED.RETIRE_SLOTS', 'INT_MISC.RECOVERY_CYCLES', 'cpu-cycles', 'CPU_CLK_UNHALTED.THREAD_ANY', 'INT_MISC.RECOVERY_CYCLES_ANY'}"
2023-06-02 21:04:59,310 WARNING: MULTIPLE GROUPS: metric "metric_TMA_..Branch_Mispredicts(%)", events "{'UOPS_ISSUED.ANY', 'UOPS_RETIRED.RETIRE_SLOTS', 'INT_MISC.RECOVERY_CYCLES', 'cpu-cycles', 'CPU_CLK_UNHALTED.THREAD_ANY', 'BR_MISP_RETIRED.ALL_BRANCHES', 'INT_MISC.RECOVERY_CYCLES_ANY', 'MACHINE_CLEARS.COUNT'}"
2023-06-02 21:04:59,312 WARNING: MULTIPLE GROUPS: metric "metric_TMA_..Machine_Clears(%)", events "{'UOPS_ISSUED.ANY', 'UOPS_RETIRED.RETIRE_SLOTS', 'INT_MISC.RECOVERY_CYCLES', 'cpu-cycles', 'CPU_CLK_UNHALTED.THREAD_ANY', 'BR_MISP_RETIRED.ALL_BRANCHES', 'INT_MISC.RECOVERY_CYCLES_ANY', 'MACHINE_CLEARS.COUNT'}"
2023-06-02 21:04:59,313 WARNING: MULTIPLE GROUPS: metric "metric_TMA_Backend_Bound(%)", events "{'UOPS_ISSUED.ANY', 'INT_MISC.RECOVERY_CYCLES', 'cpu-cycles', 'CPU_CLK_UNHALTED.THREAD_ANY', 'IDQ_UOPS_NOT_DELIVERED.CORE', 'INT_MISC.RECOVERY_CYCLES_ANY'}"
2023-06-02 21:04:59,315 WARNING: MULTIPLE GROUPS: metric "metric_TMA_..Memory_Bound(%)", events "{'EXE_ACTIVITY.1_PORTS_UTIL', 'UOPS_ISSUED.ANY', 'UOPS_RETIRED.RETIRE_SLOTS', 'CYCLE_ACTIVITY.STALLS_TOTAL', 'INT_MISC.RECOVERY_CYCLES', 'cpu-cycles', 'CPU_CLK_UNHALTED.THREAD_ANY', 'EXE_ACTIVITY.2_PORTS_UTIL', 'IDQ_UOPS_NOT_DELIVERED.CORE', 'CYCLE_ACTIVITY.STALLS_MEM_ANY', 'INT_MISC.RECOVERY_CYCLES_ANY', 'EXE_ACTIVITY.BOUND_ON_STORES'}"
2023-06-02 21:04:59,316 WARNING: MULTIPLE GROUPS: metric "metric_TMA_....L1_Bound(%)", events "{'cpu-cycles', 'CYCLE_ACTIVITY.STALLS_L1D_MISS', 'CYCLE_ACTIVITY.STALLS_MEM_ANY'}"
2023-06-02 21:04:59,317 WARNING: MULTIPLE GROUPS: metric "metric_TMA_......DTLB_Load(%)", events "{'DTLB_LOAD_MISSES.STLB_HIT:c1', 'cpu-cycles', 'DTLB_LOAD_MISSES.WALK_ACTIVE', 'CYCLE_ACTIVITY.CYCLES_MEM_ANY', 'CYCLE_ACTIVITY.CYCLES_L1D_MISS'}"
2023-06-02 21:04:59,318 WARNING: MULTIPLE GROUPS: metric "metric_TMA_......Lock_Latency(%)", events "{'L2_RQSTS.ALL_RFO', 'OFFCORE_REQUESTS_OUTSTANDING.CYCLES_WITH_DEMAND_RFO', 'MEM_INST_RETIRED.ALL_STORES', 'cpu-cycles', 'MEM_INST_RETIRED.LOCK_LOADS', 'L2_RQSTS.RFO_HIT'}"
2023-06-02 21:04:59,320 WARNING: MULTIPLE GROUPS: metric "metric_TMA_....L2_Bound(%)", events "{'MEM_LOAD_RETIRED.FB_HIT', 'cpu-cycles', 'MEM_LOAD_RETIRED.L2_HIT', 'L1D_PEND_MISS.FB_FULL:c1', 'MEM_LOAD_RETIRED.L1_MISS', 'CYCLE_ACTIVITY.STALLS_L2_MISS', 'CYCLE_ACTIVITY.STALLS_L1D_MISS'}"
2023-06-02 21:04:59,321 WARNING: MULTIPLE GROUPS: metric "metric_TMA_......Data_Sharing(%)", events "{'MEM_LOAD_L3_HIT_RETIRED.XSNP_HIT', 'MEM_LOAD_RETIRED.FB_HIT', 'ref-cycles', 'cpu-cycles', 'MEM_LOAD_RETIRED.L1_MISS', 'OCR.DEMAND_DATA_RD.L3_HIT.HIT_OTHER_CORE_FWD', 'OCR.DEMAND_DATA_RD.L3_HIT.HITM_OTHER_CORE', 'MEM_LOAD_L3_HIT_RETIRED.XSNP_HITM'}"
2023-06-02 21:04:59,324 WARNING: MULTIPLE GROUPS: metric "metric_TMA_......False_Sharing(%)", events "{'OCR.PF_L2_RFO.L3_MISS.REMOTE_HITM', 'OCR.PF_L2_RFO.L3_HIT.HITM_OTHER_CORE', 'cpu-cycles', 'OCR.DEMAND_RFO.L3_MISS.REMOTE_HITM', 'OCR.DEMAND_RFO.L3_HIT.HITM_OTHER_CORE', 'ref-cycles'}"
2023-06-02 21:04:59,326 WARNING: MULTIPLE GROUPS: metric "metric_TMA_..Core_Bound(%)", events "{'EXE_ACTIVITY.1_PORTS_UTIL', 'UOPS_ISSUED.ANY', 'UOPS_RETIRED.RETIRE_SLOTS', 'INT_MISC.RECOVERY_CYCLES', 'CYCLE_ACTIVITY.STALLS_TOTAL', 'cpu-cycles', 'CPU_CLK_UNHALTED.THREAD_ANY', 'EXE_ACTIVITY.2_PORTS_UTIL', 'IDQ_UOPS_NOT_DELIVERED.CORE', 'CYCLE_ACTIVITY.STALLS_MEM_ANY', 'INT_MISC.RECOVERY_CYCLES_ANY', 'EXE_ACTIVITY.BOUND_ON_STORES'}"
2023-06-02 21:04:59,329 WARNING: MULTIPLE GROUPS: metric "metric_TMA_....Ports_Utilization(%)", events "{'EXE_ACTIVITY.1_PORTS_UTIL', 'UOPS_RETIRED.RETIRE_SLOTS', 'CYCLE_ACTIVITY.STALLS_TOTAL', 'cpu-cycles', 'CPU_CLK_UNHALTED.THREAD_ANY', 'EXE_ACTIVITY.2_PORTS_UTIL', 'EXE_ACTIVITY.EXE_BOUND_0_PORTS', 'CYCLE_ACTIVITY.STALLS_MEM_ANY', 'ARITH.DIVIDER_ACTIVE'}"
2023-06-02 21:04:59,330 WARNING: MULTIPLE GROUPS: metric "metric_TMA_......0_Port_Utilized(%)", events "{'UOPS_EXECUTED.CORE_CYCLES_NONE', 'EXE_ACTIVITY.EXE_BOUND_0_PORTS', 'CPU_CLK_UNHALTED.THREAD_ANY'}"
2023-06-02 21:04:59,331 WARNING: MULTIPLE GROUPS: metric "metric_TMA_......1_Port_Utilized(%)", events "{'EXE_ACTIVITY.1_PORTS_UTIL', 'UOPS_EXECUTED.CORE_CYCLES_GE_2', 'UOPS_EXECUTED.CORE_CYCLES_GE_1', 'CPU_CLK_UNHALTED.THREAD_ANY'}"
2023-06-02 21:04:59,332 WARNING: MULTIPLE GROUPS: metric "metric_TMA_......2_Port_Utilized(%)", events "{'CPU_CLK_UNHALTED.THREAD_ANY', 'EXE_ACTIVITY.2_PORTS_UTIL', 'UOPS_EXECUTED.CORE_CYCLES_GE_2', 'UOPS_EXECUTED.CORE_CYCLES_GE_3'}"
2023-06-02 21:04:59,332 WARNING: MULTIPLE GROUPS: metric "metric_TMA_......3m_Ports_Utilized(%)", events "{'CPU_CLK_UNHALTED.THREAD_ANY', 'UOPS_EXECUTED.CORE_CYCLES_GE_3'}"
2023-06-02 21:04:59,333 WARNING: MULTIPLE GROUPS: metric "metric_TMA_Retiring(%)", events "{'UOPS_RETIRED.RETIRE_SLOTS', 'cpu-cycles', 'CPU_CLK_UNHALTED.THREAD_ANY'}"
2023-06-02 21:04:59,334 WARNING: MULTIPLE GROUPS: metric "metric_TMA_..Light_Operations(%)", events "{'UOPS_RETIRED.RETIRE_SLOTS', 'UOPS_RETIRED.MACRO_FUSED', 'instructions', 'cpu-cycles', 'CPU_CLK_UNHALTED.THREAD_ANY'}"
2023-06-02 21:04:59,335 WARNING: MULTIPLE GROUPS: metric "metric_TMA_....FP_Arith(%)", events "{'UOPS_EXECUTED.THREAD', 'UOPS_RETIRED.RETIRE_SLOTS', 'FP_ARITH_INST_RETIRED.SCALAR_SINGLE:u0x03', 'UOPS_EXECUTED.X87', 'cpu-cycles', 'FP_ARITH_INST_RETIRED.128B_PACKED_DOUBLE:u0xfc', 'CPU_CLK_UNHALTED.THREAD_ANY'}"
2023-06-02 21:04:59,336 WARNING: MULTIPLE GROUPS: metric "metric_TMA_......FP_Scalar(%)", events "{'UOPS_RETIRED.RETIRE_SLOTS', 'FP_ARITH_INST_RETIRED.SCALAR_SINGLE:u0x03'}"
2023-06-02 21:04:59,336 WARNING: MULTIPLE GROUPS: metric "metric_TMA_......FP_Vector(%)", events "{'UOPS_RETIRED.RETIRE_SLOTS', 'FP_ARITH_INST_RETIRED.128B_PACKED_DOUBLE:u0xfc'}"
2023-06-02 21:04:59,337 WARNING: MULTIPLE GROUPS: metric "metric_TMA_..Heavy_Operations(%)", events "{'UOPS_RETIRED.RETIRE_SLOTS', 'UOPS_RETIRED.MACRO_FUSED', 'instructions', 'cpu-cycles', 'CPU_CLK_UNHALTED.THREAD_ANY'}"
2023-06-02 21:04:59,338 WARNING: MULTIPLE GROUPS: metric "metric_TMA_..Microcode_Sequencer(%)", events "{'IDQ.MS_UOPS', 'UOPS_ISSUED.ANY', 'UOPS_RETIRED.RETIRE_SLOTS', 'CPU_CLK_UNHALTED.THREAD_ANY'}"
2023-06-02 21:04:59,338 WARNING: throwing out last sample because it was too short
2023-06-02 21:04:59,339 WARNING: 0 MISSING DATA: set()
2023-06-02 21:04:59,339 WARNING: 0 ZERO DIVISION: set()
2023-06-02 21:04:59,339 WARNING: 3 MISSING EVENTS: {'metric_core c6 residency %', 'metric_UPI Transmit utilization_% (includes control)', 'metric_package c6 residency %'}
2023-06-02 21:04:59,339 WARNING: 42 MULTIPLE GROUPS: {'metric_TMA_......FP_Vector(%)', 'metric_TMA_..Light_Operations(%)', 'metric_TMA_......2_Port_Utilized(%)', 'metric_TMA_..Memory_Bound(%)', 'metric_TMA_Bad_Speculation(%)', 'metric_TMA_......DTLB_Load(%)', 'metric_TMA_....DSB(%)', 'metric_NUMA %_Reads addressed to local DRAM', 'metric_TMA_..Frontend_Bandwidth(%)', 'metric_TMA_..Machine_Clears(%)', 'metric_TMA_....Ports_Utilization(%)', 'metric_LLC total HIT clean line forwards (per instr)', 'metric_NUMA %_Reads addressed to remote DRAM', 'metric_TMA_..Heavy_Operations(%)', 'metric_TMA_......Lock_Latency(%)', 'metric_TMA_..Core_Bound(%)', 'metric_TMA_....MITE(%)', 'metric_TMA_Retiring(%)', 'metric_Average LLC data read miss latency (in clks)', 'metric_UPI Transmit utilization_% (includes control)', 'metric_Average LLC data read miss latency for LOCAL requests (in ns)', 'metric_TMA_......1_Port_Utilized(%)', 'metric_TMA_Backend_Bound(%)', 'metric_TMA_......0_Port_Utilized(%)', 'metric_LLC data read MPI (demand+prefetch)', 'metric_TMA_......3m_Ports_Utilized(%)', 'metric_TMA_......FP_Scalar(%)', 'metric_IO_bandwidth_disk_or_network_reads (MB/sec)', 'metric_LLC code read MPI (demand+prefetch)', 'metric_IO_bandwidth_disk_or_network_writes (MB/sec)', 'metric_TMA_..Branch_Mispredicts(%)', 'metric_TMA_....Branch_Resteers(%)', 'metric_TMA_......False_Sharing(%)', 'metric_Average LLC data read miss latency for REMOTE requests (in ns)', 'metric_TMA_....FP_Arith(%)', 'metric_TMA_....L2_Bound(%)', 'metric_LLC total HITM (per instr)', 'metric_LLC MPI (includes code+data+rfo w/ prefetches)', 'metric_TMA_....L1_Bound(%)', 'metric_TMA_......Data_Sharing(%)', 'metric_TMA_..Microcode_Sequencer(%)', 'metric_TMA_..Frontend_Latency(%)'}
2023-06-02 21:04:59,361 INFO: Generated results file(s) in: /home/centos/perfspect
2023-06-02 21:04:59,361 INFO: Done!

UNC_CHA_TOR_INSERTS.IA_MISS.0x40431 event is mapped to each of the CHA devices on the system which can get listed using

ls /sys/devices/

could you please provide your logs and output of /sys/devices/?

jasonneverstop commented 1 year ago
My output is like this metric avg
metric_NUMA %_Reads addressed to local DRAM 0
metric_NUMA %_Reads addressed to remote DRAM 0 
 ./perf-collect -v -t 10
2023-06-06 09:33:24,902 INFO: nmi_watchdog disabled!
2023-06-06 09:33:24,916 INFO: Run mode: system
2023-06-06 09:33:25,192 WARNING: Perf unsupported events not counted: ['cstate_core/c6-residency/;', 'cstate_pkg/c6-residency/;']
2023-06-06 09:33:25,282 INFO: PMUs not in use
2023-06-06 09:33:25,282 INFO: perf stat -I 5000 -x , -a -o perfstat.csv -e '{cpu/event=0x28,umask=0x07,period=200003,name='CORE_POWER.LVL0_TURBO_LICENSE'/,cpu/event=0x28,umask=0x18,period=200003,name='CORE_POWER.LVL1_TURBO_LICENSE'/,cpu/event=0x28,umask=0x20,period=200003,name='CORE_POWER.LVL2_TURBO_LICENSE'/,cpu/event=0x51,umask=0x01,period=2000003,name='L1D.REPLACEMENT'/,cpu-cycles,ref-cycles,instructions}','{cpu/event=0xc3,umask=0x01,edge,period=100003,name='MACHINE_CLEARS.COUNT'/,cpu/event=0xc5,umask=0x00,period=400009,name='BR_MISP_RETIRED.ALL_BRANCHES'/,cpu/event=0x0d,umask=0x80,period=2000003,name='INT_MISC.CLEAR_RESTEER_CYCLES'/,cpu/event=0xd1,umask=0x01,period=2000003,name='MEM_LOAD_RETIRED.L1_HIT'/,cpu-cycles,ref-cycles,instructions}','{cpu/event=0x0e,umask=0x01,period=2000003,name='UOPS_ISSUED.ANY'/,cpu/event=0xc2,umask=0x02,period=2000003,name='UOPS_RETIRED.RETIRE_SLOTS'/,cpu/event=0x0d,umask=0x01,period=2000003,name='INT_MISC.RECOVERY_CYCLES_ANY'/,cpu/event=0x0d,umask=0x01,period=2000003,name='INT_MISC.RECOVERY_CYCLES'/}','{cpu-cycles,ref-cycles,instructions}','{cpu/event=0x3c,umask=0x0,period=2000003,name='CPU_CLK_UNHALTED.THREAD_ANY'/,cpu/event=0x9c,umask=0x01,period=2000003,name='IDQ_UOPS_NOT_DELIVERED.CORE'/,cpu/event=0xa3,umask=0x14,cmask=0x14,period=2000003,name='CYCLE_ACTIVITY.STALLS_MEM_ANY'/,cpu/event=0xa3,umask=0x04,cmask=0x04,period=2000003,name='CYCLE_ACTIVITY.STALLS_TOTAL'/,cpu-cycles,ref-cycles,instructions}','{cpu/event=0xa6,umask=0x02,period=2000003,name='EXE_ACTIVITY.1_PORTS_UTIL'/,cpu/event=0xa6,umask=0x04,period=2000003,name='EXE_ACTIVITY.2_PORTS_UTIL'/,cpu/event=0xa6,umask=0x40,period=2000003,name='EXE_ACTIVITY.BOUND_ON_STORES'/,cpu/event=0xa6,umask=0x01,period=2000003,name='EXE_ACTIVITY.EXE_BOUND_0_PORTS'/,cpu-cycles,ref-cycles,instructions}','{cpu/event=0xd1,umask=0x02,period=100003,name='MEM_LOAD_RETIRED.L2_HIT'/,cpu/event=0xd1,umask=0x40,period=100007,name='MEM_LOAD_RETIRED.FB_HIT'/,cpu/event=0xd1,umask=0x08,period=100003,name='MEM_LOAD_RETIRED.L1_MISS'/,cpu/event=0x48,umask=0x02,cmask=0x01,period=2000003,name='L1D_PEND_MISS.FB_FULL:c1'/,cpu-cycles,ref-cycles,instructions}','{cpu/event=0xa3,umask=0x0c,cmask=0x0c,period=2000003,name='CYCLE_ACTIVITY.STALLS_L1D_MISS'/,cpu/event=0xa3,umask=0x05,cmask=0x05,period=2000003,name='CYCLE_ACTIVITY.STALLS_L2_MISS'/,cpu/event=0xa3,umask=0x06,cmask=0x06,period=2000003,name='CYCLE_ACTIVITY.STALLS_L3_MISS'/,cpu/event=0x60,umask=0x04,cmask=0x01,period=200003,name='OFFCORE_REQUESTS_OUTSTANDING.CYCLES_WITH_DEMAND_RFO'/,cpu-cycles,ref-cycles,instructions}','{cpu/event=0x85,umask=0x0e,period=100003,name='ITLB_MISSES.WALK_COMPLETED'/,cpu/event=0x85,umask=0x10,period=100003,name='ITLB_MISSES.WALK_ACTIVE'/,cpu/event=0x08,umask=0x0e,period=100003,name='DTLB_LOAD_MISSES.WALK_COMPLETED'/,cpu/event=0x08,umask=0x10,cmask=0x01,period=100003,name='DTLB_LOAD_MISSES.WALK_ACTIVE'/,cpu-cycles,ref-cycles,instructions}','{cpu/event=0x49,umask=0x0e,period=100003,name='DTLB_STORE_MISSES.WALK_COMPLETED'/,cpu/event=0x49,umask=0x10,period=100003,name='DTLB_STORE_MISSES.WALK_ACTIVE'/,cpu/event=0x9c,umask=0x01,cmask=0x4,period=2000003,name='IDQ_UOPS_NOT_DELIVERED.CYCLES_0_UOPS_DELIV.CORE'/,cpu/event=0xe6,umask=0x01,period=100003,name='BACLEARS.ANY'/,cpu-cycles,ref-cycles,instructions}','{cpu/event=0xd0,umask=0x21,cmask=0x01,period=100007,name='MEM_INST_RETIRED.LOCK_LOADS'/,cpu/event=0x24,umask=0xe2,cmask=0x00,period=200003,name='L2_RQSTS.ALL_RFO'/,cpu/event=0xd0,umask=0x82,cmask=0x00,period=200003,name='MEM_INST_RETIRED.ALL_STORES'/,cpu/event=0x24,umask=0xc2,cmask=0x00,period=200003,name='L2_RQSTS.RFO_HIT'/,cpu-cycles,ref-cycles,instructions}','{cpu/event=0xb7,umask=0x01,offcore_rsp=0x10003C0001,period=100003,name='OCR.DEMAND_DATA_RD.L3_HIT.HITM_OTHER_CORE'/,cpu/event=0xb7,umask=0x01,offcore_rsp=0x8003C0001,period=100003,name='OCR.DEMAND_DATA_RD.L3_HIT.HIT_OTHER_CORE_FWD'/,cpu/event=0x60,umask=0x10,cmask=0x06,period=2000003,name='OFFCORE_REQUESTS_OUTSTANDING.L3_MISS_DEMAND_DATA_RD_GE_6'/,cpu/event=0x60,umask=0x10,cmask=0x01,period=2000003,name='OFFCORE_REQUESTS_OUTSTANDING.CYCLES_WITH_L3_MISS_DEMAND_DATA_RD'/,cpu-cycles,ref-cycles,instructions}','{cpu/event=0xb1,umask=0x02,cmask=0x1,period=2000003,name='UOPS_EXECUTED.CORE_CYCLES_GE_1'/,cpu/event=0xb1,umask=0x02,cmask=0x2,period=2000003,name='UOPS_EXECUTED.CORE_CYCLES_GE_2'/,cpu/event=0xb1,umask=0x02,cmask=0x3,period=2000003,name='UOPS_EXECUTED.CORE_CYCLES_GE_3'/,cpu/event=0xc2,umask=0x04,cmask=0x00,period=2000003,name='UOPS_RETIRED.MACRO_FUSED'/,cpu-cycles,ref-cycles,instructions}','{cpu/event=0xc7,umask=0x03,cmask=0x00,period=2000003,name='FP_ARITH_INST_RETIRED.SCALAR_SINGLE:u0x03'/,cpu/event=0xc7,umask=0xfc,cmask=0x00,period=2000003,name='FP_ARITH_INST_RETIRED.128B_PACKED_DOUBLE:u0xfc'/,cpu/event=0x80,umask=0x4,name='ICACHE_16B.IFDATA_STALL'/,cpu/event=0x80,umask=0x4,cmask=0x1,edge=0x1,name='ICACHE_16B.IFDATA_STALL:c1:e1'/,cpu-cycles,ref-cycles,instructions}','{cpu/event=0x24,umask=0xe4,period=200003,name='L2_RQSTS.ALL_CODE_RD'/,cpu/event=0x85,umask=0x04,period=100003,name='ITLB_MISSES.WALK_COMPLETED_2M_4M'/,cpu/event=0xf1,umask=0x1f,period=100003,name='L2_LINES_IN.ALL'/,cpu/event=0xd1,umask=0x10,period=50021,name='MEM_LOAD_RETIRED.L2_MISS'/,cpu-cycles,ref-cycles,instructions}','{cpu/event=0x24,umask=0x24,period=200003,name='L2_RQSTS.CODE_RD_MISS'/,cpu/event=0x08,umask=0x04,period=2000003,name='DTLB_LOAD_MISSES.WALK_COMPLETED_2M_4M'/,cpu/event=0x08,umask=0x02,period=2000003,name='DTLB_LOAD_MISSES.WALK_COMPLETED_4K'/,cpu/event=0x08,umask=0x08,period=2000003,name='DTLB_LOAD_MISSES.WALK_COMPLETED_1G'/,cpu-cycles,ref-cycles,instructions}','{cpu/event=0xb0,umask=0x10,period=100003,name='OFFCORE_REQUESTS.L3_MISS_DEMAND_DATA_RD'/,cpu/event=0xd1,umask=0x40,period=100007,name='MEM_LOAD_RETIRED.FB_HIT'/,cpu/event=0xa3,umask=0x10,cmask=0x16,period=2000003,name='CYCLE_ACTIVITY.CYCLES_MEM_ANY'/,cpu/event=0xa3,umask=0x08,cmask=0x08,period=2000003,name='CYCLE_ACTIVITY.CYCLES_L1D_MISS'/,cpu-cycles,ref-cycles,instructions}','{cpu/event=0x79,umask=0x30,period=2000003,name='IDQ.MS_UOPS'/,cpu/event=0x60,umask=0x10,period=2000003,name='OFFCORE_REQUESTS_OUTSTANDING.L3_MISS_DEMAND_DATA_RD'/,cpu/event=0x83,umask=0x04,period=200003,name='ICACHE_64B.IFTAG_STALL'/,cpu/event=0x08,umask=0x20,cmask=0x01,period=2000003,name='DTLB_LOAD_MISSES.STLB_HIT:c1'/,cpu-cycles,ref-cycles,instructions}','{cpu/event=0x80,umask=0x4,name='ICACHE_16B.IFDATA_STALL'/,cpu/event=0x80,umask=0x4,cmask=0x1,edge=0x1,name='ICACHE_16B.IFDATA_STALL:c1:e1'/,cpu/event=0x14,umask=0x01,period=2000003,name='ARITH.DIVIDER_ACTIVE'/,cpu/event=0xb1,umask=0x02,inv=0x1,cmask=0x1,period=2000003,name='UOPS_EXECUTED.CORE_CYCLES_NONE'/,cpu-cycles,ref-cycles,instructions}','{cpu/event=0x3c,umask=0x2,name='CPU_CLK_THREAD_UNHALTED.ONE_THREAD_ACTIVE'/,cpu/event=0x3c,umask=0x1,name='CPU_CLK_THREAD_UNHALTED.REF_XCLK_ANY'/,cpu-cycles:k,ref-cycles:k,instructions:k}','{cpu/event=0x79,umask=0x24,cmask=0x01,period=2000003,name='IDQ.ALL_MITE_CYCLES_ANY_UOPS'/,cpu/event=0x79,umask=0x24,cmask=0x04,period=2000003,name='IDQ.ALL_MITE_CYCLES_4_UOPS'/,cpu/event=0xb7,umask=0x01,offcore_rsp=0x10003C0002,period=100003,name='OCR.DEMAND_RFO.L3_HIT.HITM_OTHER_CORE'/,cpu/event=0xb7,umask=0x01,offcore_rsp=0x10003C0020,period=100003,name='OCR.PF_L2_RFO.L3_HIT.HITM_OTHER_CORE'/,cpu-cycles,ref-cycles,instructions}','{cpu/event=0x79,umask=0x18,cmask=0x01,period=2000003,name='IDQ.ALL_DSB_CYCLES_ANY_UOPS'/,cpu/event=0x79,umask=0x18,cmask=0x04,period=2000003,name='IDQ.ALL_DSB_CYCLES_4_UOPS'/,cpu/event=0xb7,umask=0x01,offcore_rsp=0x103FC00002,period=100003,name='OCR.DEMAND_RFO.L3_MISS.REMOTE_HITM'/,cpu/event=0xb7,umask=0x01,offcore_rsp=0x103FC00020,period=100003,name='OCR.PF_L2_RFO.L3_MISS.REMOTE_HITM'/,cpu-cycles,ref-cycles,instructions}','{cpu/event=0xd2,umask=0x02,cmask=0x00,period=20011,name='MEM_LOAD_L3_HIT_RETIRED.XSNP_HIT'/,cpu/event=0xd2,umask=0x04,cmask=0x00,period=20011,name='MEM_LOAD_L3_HIT_RETIRED.XSNP_HITM'/,cpu/event=0xb7,umask=0x01,offcore_rsp=0x3F840007F7,name='OCR.ALL_READS.L3_MISS_LOCAL_DRAM.ANY_SNOOP'/,cpu/event=0xb7,umask=0x01,offcore_rsp=0x3FB80007F7,name='OCR.ALL_READS.L3_MISS_LOCAL_DRAM.ANY_SNOOP_ocr_msr_3fB80007f7'/}','{cpu-cycles,ref-cycles,instructions}','{cpu/event=0xb1,umask=0x10,cmask=0x00,period=2000003,name='UOPS_EXECUTED.X87'/,cpu/event=0xb1,umask=0x01,cmask=0x00,period=2000003,name='UOPS_EXECUTED.THREAD'/,cpu/event=0xb7,umask=0x01,offcore_rsp=0x103FC007F7,name='OCR.ALL_READS.L3_MISS.REMOTE_HITM'/,cpu/event=0xb7,umask=0x01,offcore_rsp=0x083FC007F7,name='OCR.ALL_READS.L3_MISS.REMOTE_HIT_FORWARD'/}','{cpu-cycles,ref-cycles,instructions}','{power/energy-pkg/,power/energy-ram/}','{uncore_imc_0/event=0x04,umask=0x03,name='UNC_M_CAS_COUNT.RD.0'/,uncore_imc_0/event=0x04,umask=0x0c,name='UNC_M_CAS_COUNT.WR.0'/}','{uncore_imc_1/event=0x04,umask=0x03,name='UNC_M_CAS_COUNT.RD.1'/,uncore_imc_1/event=0x04,umask=0x0c,name='UNC_M_CAS_COUNT.WR.1'/}','{uncore_imc_2/event=0x04,umask=0x03,name='UNC_M_CAS_COUNT.RD.2'/,uncore_imc_2/event=0x04,umask=0x0c,name='UNC_M_CAS_COUNT.WR.2'/}','{uncore_imc_3/event=0x04,umask=0x03,name='UNC_M_CAS_COUNT.RD.3'/,uncore_imc_3/event=0x04,umask=0x0c,name='UNC_M_CAS_COUNT.WR.3'/}','{uncore_imc_4/event=0x04,umask=0x03,name='UNC_M_CAS_COUNT.RD.4'/,uncore_imc_4/event=0x04,umask=0x0c,name='UNC_M_CAS_COUNT.WR.4'/}','{uncore_imc_5/event=0x04,umask=0x03,name='UNC_M_CAS_COUNT.RD.5'/,uncore_imc_5/event=0x04,umask=0x0c,name='UNC_M_CAS_COUNT.WR.5'/}','{uncore_cha_0/event=0x0,umask=0x0,name='UNC_CHA_CLOCKTICKS.0'/,uncore_cha_0/event=0x36,umask=0x21,config1=0x4043300000000,name='UNC_CHA_TOR_OCCUPANCY.IA_MISS.0x40433.0'/,uncore_cha_0/event=0x35,umask=0x21,config1=0x4043300000000,name='UNC_CHA_TOR_INSERTS.IA_MISS.0x40433.0'/}','{uncore_cha_1/event=0x0,umask=0x0,name='UNC_CHA_CLOCKTICKS.1'/,uncore_cha_1/event=0x36,umask=0x21,config1=0x4043300000000,name='UNC_CHA_TOR_OCCUPANCY.IA_MISS.0x40433.1'/,uncore_cha_1/event=0x35,umask=0x21,config1=0x4043300000000,name='UNC_CHA_TOR_INSERTS.IA_MISS.0x40433.1'/}','{uncore_cha_2/event=0x0,umask=0x0,name='UNC_CHA_CLOCKTICKS.2'/,uncore_cha_2/event=0x36,umask=0x21,config1=0x4043300000000,name='UNC_CHA_TOR_OCCUPANCY.IA_MISS.0x40433.2'/,uncore_cha_2/event=0x35,umask=0x21,config1=0x4043300000000,name='UNC_CHA_TOR_INSERTS.IA_MISS.0x40433.2'/}','{uncore_cha_3/event=0x0,umask=0x0,name='UNC_CHA_CLOCKTICKS.3'/,uncore_cha_3/event=0x36,umask=0x21,config1=0x4043300000000,name='UNC_CHA_TOR_OCCUPANCY.IA_MISS.0x40433.3'/,uncore_cha_3/event=0x35,umask=0x21,config1=0x4043300000000,name='UNC_CHA_TOR_INSERTS.IA_MISS.0x40433.3'/}','{uncore_cha_4/event=0x0,umask=0x0,name='UNC_CHA_CLOCKTICKS.4'/,uncore_cha_4/event=0x36,umask=0x21,config1=0x4043300000000,name='UNC_CHA_TOR_OCCUPANCY.IA_MISS.0x40433.4'/,uncore_cha_4/event=0x35,umask=0x21,config1=0x4043300000000,name='UNC_CHA_TOR_INSERTS.IA_MISS.0x40433.4'/}','{uncore_cha_5/event=0x0,umask=0x0,name='UNC_CHA_CLOCKTICKS.5'/,uncore_cha_5/event=0x36,umask=0x21,config1=0x4043300000000,name='UNC_CHA_TOR_OCCUPANCY.IA_MISS.0x40433.5'/,uncore_cha_5/event=0x35,umask=0x21,config1=0x4043300000000,name='UNC_CHA_TOR_INSERTS.IA_MISS.0x40433.5'/}','{uncore_cha_6/event=0x0,umask=0x0,name='UNC_CHA_CLOCKTICKS.6'/,uncore_cha_6/event=0x36,umask=0x21,config1=0x4043300000000,name='UNC_CHA_TOR_OCCUPANCY.IA_MISS.0x40433.6'/,uncore_cha_6/event=0x35,umask=0x21,config1=0x4043300000000,name='UNC_CHA_TOR_INSERTS.IA_MISS.0x40433.6'/}','{uncore_cha_7/event=0x0,umask=0x0,name='UNC_CHA_CLOCKTICKS.7'/,uncore_cha_7/event=0x36,umask=0x21,config1=0x4043300000000,name='UNC_CHA_TOR_OCCUPANCY.IA_MISS.0x40433.7'/,uncore_cha_7/event=0x35,umask=0x21,config1=0x4043300000000,name='UNC_CHA_TOR_INSERTS.IA_MISS.0x40433.7'/}','{uncore_cha_8/event=0x0,umask=0x0,name='UNC_CHA_CLOCKTICKS.8'/,uncore_cha_8/event=0x36,umask=0x21,config1=0x4043300000000,name='UNC_CHA_TOR_OCCUPANCY.IA_MISS.0x40433.8'/,uncore_cha_8/event=0x35,umask=0x21,config1=0x4043300000000,name='UNC_CHA_TOR_INSERTS.IA_MISS.0x40433.8'/}','{uncore_cha_9/event=0x0,umask=0x0,name='UNC_CHA_CLOCKTICKS.9'/,uncore_cha_9/event=0x36,umask=0x21,config1=0x4043300000000,name='UNC_CHA_TOR_OCCUPANCY.IA_MISS.0x40433.9'/,uncore_cha_9/event=0x35,umask=0x21,config1=0x4043300000000,name='UNC_CHA_TOR_INSERTS.IA_MISS.0x40433.9'/}','{uncore_cha_10/event=0x0,umask=0x0,name='UNC_CHA_CLOCKTICKS.10'/,uncore_cha_10/event=0x36,umask=0x21,config1=0x4043300000000,name='UNC_CHA_TOR_OCCUPANCY.IA_MISS.0x40433.10'/,uncore_cha_10/event=0x35,umask=0x21,config1=0x4043300000000,name='UNC_CHA_TOR_INSERTS.IA_MISS.0x40433.10'/}','{uncore_cha_11/event=0x0,umask=0x0,name='UNC_CHA_CLOCKTICKS.11'/,uncore_cha_11/event=0x36,umask=0x21,config1=0x4043300000000,name='UNC_CHA_TOR_OCCUPANCY.IA_MISS.0x40433.11'/,uncore_cha_11/event=0x35,umask=0x21,config1=0x4043300000000,name='UNC_CHA_TOR_INSERTS.IA_MISS.0x40433.11'/}','{uncore_cha_12/event=0x0,umask=0x0,name='UNC_CHA_CLOCKTICKS.12'/,uncore_cha_12/event=0x36,umask=0x21,config1=0x4043300000000,name='UNC_CHA_TOR_OCCUPANCY.IA_MISS.0x40433.12'/,uncore_cha_12/event=0x35,umask=0x21,config1=0x4043300000000,name='UNC_CHA_TOR_INSERTS.IA_MISS.0x40433.12'/}','{uncore_cha_13/event=0x0,umask=0x0,name='UNC_CHA_CLOCKTICKS.13'/,uncore_cha_13/event=0x36,umask=0x21,config1=0x4043300000000,name='UNC_CHA_TOR_OCCUPANCY.IA_MISS.0x40433.13'/,uncore_cha_13/event=0x35,umask=0x21,config1=0x4043300000000,name='UNC_CHA_TOR_INSERTS.IA_MISS.0x40433.13'/}','{uncore_cha_14/event=0x0,umask=0x0,name='UNC_CHA_CLOCKTICKS.14'/,uncore_cha_14/event=0x36,umask=0x21,config1=0x4043300000000,name='UNC_CHA_TOR_OCCUPANCY.IA_MISS.0x40433.14'/,uncore_cha_14/event=0x35,umask=0x21,config1=0x4043300000000,name='UNC_CHA_TOR_INSERTS.IA_MISS.0x40433.14'/}','{uncore_cha_15/event=0x0,umask=0x0,name='UNC_CHA_CLOCKTICKS.15'/,uncore_cha_15/event=0x36,umask=0x21,config1=0x4043300000000,name='UNC_CHA_TOR_OCCUPANCY.IA_MISS.0x40433.15'/,uncore_cha_15/event=0x35,umask=0x21,config1=0x4043300000000,name='UNC_CHA_TOR_INSERTS.IA_MISS.0x40433.15'/}','{uncore_cha_16/event=0x0,umask=0x0,name='UNC_CHA_CLOCKTICKS.16'/,uncore_cha_16/event=0x36,umask=0x21,config1=0x4043300000000,name='UNC_CHA_TOR_OCCUPANCY.IA_MISS.0x40433.16'/,uncore_cha_16/event=0x35,umask=0x21,config1=0x4043300000000,name='UNC_CHA_TOR_INSERTS.IA_MISS.0x40433.16'/}','{uncore_cha_17/event=0x0,umask=0x0,name='UNC_CHA_CLOCKTICKS.17'/,uncore_cha_17/event=0x36,umask=0x21,config1=0x4043300000000,name='UNC_CHA_TOR_OCCUPANCY.IA_MISS.0x40433.17'/,uncore_cha_17/event=0x35,umask=0x21,config1=0x4043300000000,name='UNC_CHA_TOR_INSERTS.IA_MISS.0x40433.17'/}','{uncore_cha_18/event=0x0,umask=0x0,name='UNC_CHA_CLOCKTICKS.18'/,uncore_cha_18/event=0x36,umask=0x21,config1=0x4043300000000,name='UNC_CHA_TOR_OCCUPANCY.IA_MISS.0x40433.18'/,uncore_cha_18/event=0x35,umask=0x21,config1=0x4043300000000,name='UNC_CHA_TOR_INSERTS.IA_MISS.0x40433.18'/}','{uncore_cha_19/event=0x0,umask=0x0,name='UNC_CHA_CLOCKTICKS.19'/,uncore_cha_19/event=0x36,umask=0x21,config1=0x4043300000000,name='UNC_CHA_TOR_OCCUPANCY.IA_MISS.0x40433.19'/,uncore_cha_19/event=0x35,umask=0x21,config1=0x4043300000000,name='UNC_CHA_TOR_INSERTS.IA_MISS.0x40433.19'/}','{uncore_cha_20/event=0x0,umask=0x0,name='UNC_CHA_CLOCKTICKS.20'/,uncore_cha_20/event=0x36,umask=0x21,config1=0x4043300000000,name='UNC_CHA_TOR_OCCUPANCY.IA_MISS.0x40433.20'/,uncore_cha_20/event=0x35,umask=0x21,config1=0x4043300000000,name='UNC_CHA_TOR_INSERTS.IA_MISS.0x40433.20'/}','{uncore_cha_21/event=0x0,umask=0x0,name='UNC_CHA_CLOCKTICKS.21'/,uncore_cha_21/event=0x36,umask=0x21,config1=0x4043300000000,name='UNC_CHA_TOR_OCCUPANCY.IA_MISS.0x40433.21'/,uncore_cha_21/event=0x35,umask=0x21,config1=0x4043300000000,name='UNC_CHA_TOR_INSERTS.IA_MISS.0x40433.21'/}','{uncore_cha_22/event=0x0,umask=0x0,name='UNC_CHA_CLOCKTICKS.22'/,uncore_cha_22/event=0x36,umask=0x21,config1=0x4043300000000,name='UNC_CHA_TOR_OCCUPANCY.IA_MISS.0x40433.22'/,uncore_cha_22/event=0x35,umask=0x21,config1=0x4043300000000,name='UNC_CHA_TOR_INSERTS.IA_MISS.0x40433.22'/}','{uncore_cha_23/event=0x0,umask=0x0,name='UNC_CHA_CLOCKTICKS.23'/,uncore_cha_23/event=0x36,umask=0x21,config1=0x4043300000000,name='UNC_CHA_TOR_OCCUPANCY.IA_MISS.0x40433.23'/,uncore_cha_23/event=0x35,umask=0x21,config1=0x4043300000000,name='UNC_CHA_TOR_INSERTS.IA_MISS.0x40433.23'/}','{uncore_cha_24/event=0x0,umask=0x0,name='UNC_CHA_CLOCKTICKS.24'/,uncore_cha_24/event=0x36,umask=0x21,config1=0x4043300000000,name='UNC_CHA_TOR_OCCUPANCY.IA_MISS.0x40433.24'/,uncore_cha_24/event=0x35,umask=0x21,config1=0x4043300000000,name='UNC_CHA_TOR_INSERTS.IA_MISS.0x40433.24'/}','{uncore_cha_25/event=0x0,umask=0x0,name='UNC_CHA_CLOCKTICKS.25'/,uncore_cha_25/event=0x36,umask=0x21,config1=0x4043300000000,name='UNC_CHA_TOR_OCCUPANCY.IA_MISS.0x40433.25'/,uncore_cha_25/event=0x35,umask=0x21,config1=0x4043300000000,name='UNC_CHA_TOR_INSERTS.IA_MISS.0x40433.25'/}','{uncore_cha_26/event=0x0,umask=0x0,name='UNC_CHA_CLOCKTICKS.26'/,uncore_cha_26/event=0x36,umask=0x21,config1=0x4043300000000,name='UNC_CHA_TOR_OCCUPANCY.IA_MISS.0x40433.26'/,uncore_cha_26/event=0x35,umask=0x21,config1=0x4043300000000,name='UNC_CHA_TOR_INSERTS.IA_MISS.0x40433.26'/}','{uncore_cha_27/event=0x0,umask=0x0,name='UNC_CHA_CLOCKTICKS.27'/,uncore_cha_27/event=0x36,umask=0x21,config1=0x4043300000000,name='UNC_CHA_TOR_OCCUPANCY.IA_MISS.0x40433.27'/,uncore_cha_27/event=0x35,umask=0x21,config1=0x4043300000000,name='UNC_CHA_TOR_INSERTS.IA_MISS.0x40433.27'/}','{uncore_cha_0/event=0x35,umask=0x21,config1=0x4043200000000,name='UNC_CHA_TOR_INSERTS.IA_MISS.0x40432.0'/,uncore_cha_0/event=0x36,umask=0x21,config1=0x4043200000000,name='UNC_CHA_TOR_OCCUPANCY.IA_MISS.0x40432.0'/}','{uncore_cha_1/event=0x35,umask=0x21,config1=0x4043200000000,name='UNC_CHA_TOR_INSERTS.IA_MISS.0x40432.1'/,uncore_cha_1/event=0x36,umask=0x21,config1=0x4043200000000,name='UNC_CHA_TOR_OCCUPANCY.IA_MISS.0x40432.1'/}','{uncore_cha_2/event=0x35,umask=0x21,config1=0x4043200000000,name='UNC_CHA_TOR_INSERTS.IA_MISS.0x40432.2'/,uncore_cha_2/event=0x36,umask=0x21,config1=0x4043200000000,name='UNC_CHA_TOR_OCCUPANCY.IA_MISS.0x40432.2'/}','{uncore_cha_3/event=0x35,umask=0x21,config1=0x4043200000000,name='UNC_CHA_TOR_INSERTS.IA_MISS.0x40432.3'/,uncore_cha_3/event=0x36,umask=0x21,config1=0x4043200000000,name='UNC_CHA_TOR_OCCUPANCY.IA_MISS.0x40432.3'/}','{uncore_cha_4/event=0x35,umask=0x21,config1=0x4043200000000,name='UNC_CHA_TOR_INSERTS.IA_MISS.0x40432.4'/,uncore_cha_4/event=0x36,umask=0x21,config1=0x4043200000000,name='UNC_CHA_TOR_OCCUPANCY.IA_MISS.0x40432.4'/}','{uncore_cha_5/event=0x35,umask=0x21,config1=0x4043200000000,name='UNC_CHA_TOR_INSERTS.IA_MISS.0x40432.5'/,uncore_cha_5/event=0x36,umask=0x21,config1=0x4043200000000,name='UNC_CHA_TOR_OCCUPANCY.IA_MISS.0x40432.5'/}','{uncore_cha_6/event=0x35,umask=0x21,config1=0x4043200000000,name='UNC_CHA_TOR_INSERTS.IA_MISS.0x40432.6'/,uncore_cha_6/event=0x36,umask=0x21,config1=0x4043200000000,name='UNC_CHA_TOR_OCCUPANCY.IA_MISS.0x40432.6'/}','{uncore_cha_7/event=0x35,umask=0x21,config1=0x4043200000000,name='UNC_CHA_TOR_INSERTS.IA_MISS.0x40432.7'/,uncore_cha_7/event=0x36,umask=0x21,config1=0x4043200000000,name='UNC_CHA_TOR_OCCUPANCY.IA_MISS.0x40432.7'/}','{uncore_cha_8/event=0x35,umask=0x21,config1=0x4043200000000,name='UNC_CHA_TOR_INSERTS.IA_MISS.0x40432.8'/,uncore_cha_8/event=0x36,umask=0x21,config1=0x4043200000000,name='UNC_CHA_TOR_OCCUPANCY.IA_MISS.0x40432.8'/}','{uncore_cha_9/event=0x35,umask=0x21,config1=0x4043200000000,name='UNC_CHA_TOR_INSERTS.IA_MISS.0x40432.9'/,uncore_cha_9/event=0x36,umask=0x21,config1=0x4043200000000,name='UNC_CHA_TOR_OCCUPANCY.IA_MISS.0x40432.9'/}','{uncore_cha_10/event=0x35,umask=0x21,config1=0x4043200000000,name='UNC_CHA_TOR_INSERTS.IA_MISS.0x40432.10'/,uncore_cha_10/event=0x36,umask=0x21,config1=0x4043200000000,name='UNC_CHA_TOR_OCCUPANCY.IA_MISS.0x40432.10'/}','{uncore_cha_11/event=0x35,umask=0x21,config1=0x4043200000000,name='UNC_CHA_TOR_INSERTS.IA_MISS.0x40432.11'/,uncore_cha_11/event=0x36,umask=0x21,config1=0x4043200000000,name='UNC_CHA_TOR_OCCUPANCY.IA_MISS.0x40432.11'/}','{uncore_cha_12/event=0x35,umask=0x21,config1=0x4043200000000,name='UNC_CHA_TOR_INSERTS.IA_MISS.0x40432.12'/,uncore_cha_12/event=0x36,umask=0x21,config1=0x4043200000000,name='UNC_CHA_TOR_OCCUPANCY.IA_MISS.0x40432.12'/}','{uncore_cha_13/event=0x35,umask=0x21,config1=0x4043200000000,name='UNC_CHA_TOR_INSERTS.IA_MISS.0x40432.13'/,uncore_cha_13/event=0x36,umask=0x21,config1=0x4043200000000,name='UNC_CHA_TOR_OCCUPANCY.IA_MISS.0x40432.13'/}','{uncore_cha_14/event=0x35,umask=0x21,config1=0x4043200000000,name='UNC_CHA_TOR_INSERTS.IA_MISS.0x40432.14'/,uncore_cha_14/event=0x36,umask=0x21,config1=0x4043200000000,name='UNC_CHA_TOR_OCCUPANCY.IA_MISS.0x40432.14'/}','{uncore_cha_15/event=0x35,umask=0x21,config1=0x4043200000000,name='UNC_CHA_TOR_INSERTS.IA_MISS.0x40432.15'/,uncore_cha_15/event=0x36,umask=0x21,config1=0x4043200000000,name='UNC_CHA_TOR_OCCUPANCY.IA_MISS.0x40432.15'/}','{uncore_cha_16/event=0x35,umask=0x21,config1=0x4043200000000,name='UNC_CHA_TOR_INSERTS.IA_MISS.0x40432.16'/,uncore_cha_16/event=0x36,umask=0x21,config1=0x4043200000000,name='UNC_CHA_TOR_OCCUPANCY.IA_MISS.0x40432.16'/}','{uncore_cha_17/event=0x35,umask=0x21,config1=0x4043200000000,name='UNC_CHA_TOR_INSERTS.IA_MISS.0x40432.17'/,uncore_cha_17/event=0x36,umask=0x21,config1=0x4043200000000,name='UNC_CHA_TOR_OCCUPANCY.IA_MISS.0x40432.17'/}','{uncore_cha_18/event=0x35,umask=0x21,config1=0x4043200000000,name='UNC_CHA_TOR_INSERTS.IA_MISS.0x40432.18'/,uncore_cha_18/event=0x36,umask=0x21,config1=0x4043200000000,name='UNC_CHA_TOR_OCCUPANCY.IA_MISS.0x40432.18'/}','{uncore_cha_19/event=0x35,umask=0x21,config1=0x4043200000000,name='UNC_CHA_TOR_INSERTS.IA_MISS.0x40432.19'/,uncore_cha_19/event=0x36,umask=0x21,config1=0x4043200000000,name='UNC_CHA_TOR_OCCUPANCY.IA_MISS.0x40432.19'/}','{uncore_cha_20/event=0x35,umask=0x21,config1=0x4043200000000,name='UNC_CHA_TOR_INSERTS.IA_MISS.0x40432.20'/,uncore_cha_20/event=0x36,umask=0x21,config1=0x4043200000000,name='UNC_CHA_TOR_OCCUPANCY.IA_MISS.0x40432.20'/}','{uncore_cha_21/event=0x35,umask=0x21,config1=0x4043200000000,name='UNC_CHA_TOR_INSERTS.IA_MISS.0x40432.21'/,uncore_cha_21/event=0x36,umask=0x21,config1=0x4043200000000,name='UNC_CHA_TOR_OCCUPANCY.IA_MISS.0x40432.21'/}','{uncore_cha_22/event=0x35,umask=0x21,config1=0x4043200000000,name='UNC_CHA_TOR_INSERTS.IA_MISS.0x40432.22'/,uncore_cha_22/event=0x36,umask=0x21,config1=0x4043200000000,name='UNC_CHA_TOR_OCCUPANCY.IA_MISS.0x40432.22'/}','{uncore_cha_23/event=0x35,umask=0x21,config1=0x4043200000000,name='UNC_CHA_TOR_INSERTS.IA_MISS.0x40432.23'/,uncore_cha_23/event=0x36,umask=0x21,config1=0x4043200000000,name='UNC_CHA_TOR_OCCUPANCY.IA_MISS.0x40432.23'/}','{uncore_cha_24/event=0x35,umask=0x21,config1=0x4043200000000,name='UNC_CHA_TOR_INSERTS.IA_MISS.0x40432.24'/,uncore_cha_24/event=0x36,umask=0x21,config1=0x4043200000000,name='UNC_CHA_TOR_OCCUPANCY.IA_MISS.0x40432.24'/}','{uncore_cha_25/event=0x35,umask=0x21,config1=0x4043200000000,name='UNC_CHA_TOR_INSERTS.IA_MISS.0x40432.25'/,uncore_cha_25/event=0x36,umask=0x21,config1=0x4043200000000,name='UNC_CHA_TOR_OCCUPANCY.IA_MISS.0x40432.25'/}','{uncore_cha_26/event=0x35,umask=0x21,config1=0x4043200000000,name='UNC_CHA_TOR_INSERTS.IA_MISS.0x40432.26'/,uncore_cha_26/event=0x36,umask=0x21,config1=0x4043200000000,name='UNC_CHA_TOR_OCCUPANCY.IA_MISS.0x40432.26'/}','{uncore_cha_27/event=0x35,umask=0x21,config1=0x4043200000000,name='UNC_CHA_TOR_INSERTS.IA_MISS.0x40432.27'/,uncore_cha_27/event=0x36,umask=0x21,config1=0x4043200000000,name='UNC_CHA_TOR_OCCUPANCY.IA_MISS.0x40432.27'/}','{uncore_cha_0/event=0x35,umask=0x21,config1=0x4043100000000,name='UNC_CHA_TOR_INSERTS.IA_MISS.0x40431.0'/,uncore_cha_0/event=0x36,umask=0x21,config1=0x4043100000000,name='UNC_CHA_TOR_OCCUPANCY.IA_MISS.0x40431.0'/}','{uncore_cha_1/event=0x35,umask=0x21,config1=0x4043100000000,name='UNC_CHA_TOR_INSERTS.IA_MISS.0x40431.1'/,uncore_cha_1/event=0x36,umask=0x21,config1=0x4043100000000,name='UNC_CHA_TOR_OCCUPANCY.IA_MISS.0x40431.1'/}','{uncore_cha_2/event=0x35,umask=0x21,config1=0x4043100000000,name='UNC_CHA_TOR_INSERTS.IA_MISS.0x40431.2'/,uncore_cha_2/event=0x36,umask=0x21,config1=0x4043100000000,name='UNC_CHA_TOR_OCCUPANCY.IA_MISS.0x40431.2'/}','{uncore_cha_3/event=0x35,umask=0x21,config1=0x4043100000000,name='UNC_CHA_TOR_INSERTS.IA_MISS.0x40431.3'/,uncore_cha_3/event=0x36,umask=0x21,config1=0x4043100000000,name='UNC_CHA_TOR_OCCUPANCY.IA_MISS.0x40431.3'/}','{uncore_cha_4/event=0x35,umask=0x21,config1=0x4043100000000,name='UNC_CHA_TOR_INSERTS.IA_MISS.0x40431.4'/,uncore_cha_4/event=0x36,umask=0x21,config1=0x4043100000000,name='UNC_CHA_TOR_OCCUPANCY.IA_MISS.0x40431.4'/}','{uncore_cha_5/event=0x35,umask=0x21,config1=0x4043100000000,name='UNC_CHA_TOR_INSERTS.IA_MISS.0x40431.5'/,uncore_cha_5/event=0x36,umask=0x21,config1=0x4043100000000,name='UNC_CHA_TOR_OCCUPANCY.IA_MISS.0x40431.5'/}','{uncore_cha_6/event=0x35,umask=0x21,config1=0x4043100000000,name='UNC_CHA_TOR_INSERTS.IA_MISS.0x40431.6'/,uncore_cha_6/event=0x36,umask=0x21,config1=0x4043100000000,name='UNC_CHA_TOR_OCCUPANCY.IA_MISS.0x40431.6'/}','{uncore_cha_7/event=0x35,umask=0x21,config1=0x4043100000000,name='UNC_CHA_TOR_INSERTS.IA_MISS.0x40431.7'/,uncore_cha_7/event=0x36,umask=0x21,config1=0x4043100000000,name='UNC_CHA_TOR_OCCUPANCY.IA_MISS.0x40431.7'/}','{uncore_cha_8/event=0x35,umask=0x21,config1=0x4043100000000,name='UNC_CHA_TOR_INSERTS.IA_MISS.0x40431.8'/,uncore_cha_8/event=0x36,umask=0x21,config1=0x4043100000000,name='UNC_CHA_TOR_OCCUPANCY.IA_MISS.0x40431.8'/}','{uncore_cha_9/event=0x35,umask=0x21,config1=0x4043100000000,name='UNC_CHA_TOR_INSERTS.IA_MISS.0x40431.9'/,uncore_cha_9/event=0x36,umask=0x21,config1=0x4043100000000,name='UNC_CHA_TOR_OCCUPANCY.IA_MISS.0x40431.9'/}','{uncore_cha_10/event=0x35,umask=0x21,config1=0x4043100000000,name='UNC_CHA_TOR_INSERTS.IA_MISS.0x40431.10'/,uncore_cha_10/event=0x36,umask=0x21,config1=0x4043100000000,name='UNC_CHA_TOR_OCCUPANCY.IA_MISS.0x40431.10'/}','{uncore_cha_11/event=0x35,umask=0x21,config1=0x4043100000000,name='UNC_CHA_TOR_INSERTS.IA_MISS.0x40431.11'/,uncore_cha_11/event=0x36,umask=0x21,config1=0x4043100000000,name='UNC_CHA_TOR_OCCUPANCY.IA_MISS.0x40431.11'/}','{uncore_cha_12/event=0x35,umask=0x21,config1=0x4043100000000,name='UNC_CHA_TOR_INSERTS.IA_MISS.0x40431.12'/,uncore_cha_12/event=0x36,umask=0x21,config1=0x4043100000000,name='UNC_CHA_TOR_OCCUPANCY.IA_MISS.0x40431.12'/}','{uncore_cha_13/event=0x35,umask=0x21,config1=0x4043100000000,name='UNC_CHA_TOR_INSERTS.IA_MISS.0x40431.13'/,uncore_cha_13/event=0x36,umask=0x21,config1=0x4043100000000,name='UNC_CHA_TOR_OCCUPANCY.IA_MISS.0x40431.13'/}','{uncore_cha_14/event=0x35,umask=0x21,config1=0x4043100000000,name='UNC_CHA_TOR_INSERTS.IA_MISS.0x40431.14'/,uncore_cha_14/event=0x36,umask=0x21,config1=0x4043100000000,name='UNC_CHA_TOR_OCCUPANCY.IA_MISS.0x40431.14'/}','{uncore_cha_15/event=0x35,umask=0x21,config1=0x4043100000000,name='UNC_CHA_TOR_INSERTS.IA_MISS.0x40431.15'/,uncore_cha_15/event=0x36,umask=0x21,config1=0x4043100000000,name='UNC_CHA_TOR_OCCUPANCY.IA_MISS.0x40431.15'/}','{uncore_cha_16/event=0x35,umask=0x21,config1=0x4043100000000,name='UNC_CHA_TOR_INSERTS.IA_MISS.0x40431.16'/,uncore_cha_16/event=0x36,umask=0x21,config1=0x4043100000000,name='UNC_CHA_TOR_OCCUPANCY.IA_MISS.0x40431.16'/}','{uncore_cha_17/event=0x35,umask=0x21,config1=0x4043100000000,name='UNC_CHA_TOR_INSERTS.IA_MISS.0x40431.17'/,uncore_cha_17/event=0x36,umask=0x21,config1=0x4043100000000,name='UNC_CHA_TOR_OCCUPANCY.IA_MISS.0x40431.17'/}','{uncore_cha_18/event=0x35,umask=0x21,config1=0x4043100000000,name='UNC_CHA_TOR_INSERTS.IA_MISS.0x40431.18'/,uncore_cha_18/event=0x36,umask=0x21,config1=0x4043100000000,name='UNC_CHA_TOR_OCCUPANCY.IA_MISS.0x40431.18'/}','{uncore_cha_19/event=0x35,umask=0x21,config1=0x4043100000000,name='UNC_CHA_TOR_INSERTS.IA_MISS.0x40431.19'/,uncore_cha_19/event=0x36,umask=0x21,config1=0x4043100000000,name='UNC_CHA_TOR_OCCUPANCY.IA_MISS.0x40431.19'/}','{uncore_cha_20/event=0x35,umask=0x21,config1=0x4043100000000,name='UNC_CHA_TOR_INSERTS.IA_MISS.0x40431.20'/,uncore_cha_20/event=0x36,umask=0x21,config1=0x4043100000000,name='UNC_CHA_TOR_OCCUPANCY.IA_MISS.0x40431.20'/}','{uncore_cha_21/event=0x35,umask=0x21,config1=0x4043100000000,name='UNC_CHA_TOR_INSERTS.IA_MISS.0x40431.21'/,uncore_cha_21/event=0x36,umask=0x21,config1=0x4043100000000,name='UNC_CHA_TOR_OCCUPANCY.IA_MISS.0x40431.21'/}','{uncore_cha_22/event=0x35,umask=0x21,config1=0x4043100000000,name='UNC_CHA_TOR_INSERTS.IA_MISS.0x40431.22'/,uncore_cha_22/event=0x36,umask=0x21,config1=0x4043100000000,name='UNC_CHA_TOR_OCCUPANCY.IA_MISS.0x40431.22'/}','{uncore_cha_23/event=0x35,umask=0x21,config1=0x4043100000000,name='UNC_CHA_TOR_INSERTS.IA_MISS.0x40431.23'/,uncore_cha_23/event=0x36,umask=0x21,config1=0x4043100000000,name='UNC_CHA_TOR_OCCUPANCY.IA_MISS.0x40431.23'/}','{uncore_cha_24/event=0x35,umask=0x21,config1=0x4043100000000,name='UNC_CHA_TOR_INSERTS.IA_MISS.0x40431.24'/,uncore_cha_24/event=0x36,umask=0x21,config1=0x4043100000000,name='UNC_CHA_TOR_OCCUPANCY.IA_MISS.0x40431.24'/}','{uncore_cha_25/event=0x35,umask=0x21,config1=0x4043100000000,name='UNC_CHA_TOR_INSERTS.IA_MISS.0x40431.25'/,uncore_cha_25/event=0x36,umask=0x21,config1=0x4043100000000,name='UNC_CHA_TOR_OCCUPANCY.IA_MISS.0x40431.25'/}','{uncore_cha_26/event=0x35,umask=0x21,config1=0x4043100000000,name='UNC_CHA_TOR_INSERTS.IA_MISS.0x40431.26'/,uncore_cha_26/event=0x36,umask=0x21,config1=0x4043100000000,name='UNC_CHA_TOR_OCCUPANCY.IA_MISS.0x40431.26'/}','{uncore_cha_27/event=0x35,umask=0x21,config1=0x4043100000000,name='UNC_CHA_TOR_INSERTS.IA_MISS.0x40431.27'/,uncore_cha_27/event=0x36,umask=0x21,config1=0x4043100000000,name='UNC_CHA_TOR_OCCUPANCY.IA_MISS.0x40431.27'/}','{uncore_cha_0/event=0x35,umask=0x21,config1=0x12CC023300000000,name='UNC_CHA_TOR_INSERTS.IA_MISS.0x12CC0233.0'/}','{uncore_cha_1/event=0x35,umask=0x21,config1=0x12CC023300000000,name='UNC_CHA_TOR_INSERTS.IA_MISS.0x12CC0233.1'/}','{uncore_cha_2/event=0x35,umask=0x21,config1=0x12CC023300000000,name='UNC_CHA_TOR_INSERTS.IA_MISS.0x12CC0233.2'/}','{uncore_cha_3/event=0x35,umask=0x21,config1=0x12CC023300000000,name='UNC_CHA_TOR_INSERTS.IA_MISS.0x12CC0233.3'/}','{uncore_cha_4/event=0x35,umask=0x21,config1=0x12CC023300000000,name='UNC_CHA_TOR_INSERTS.IA_MISS.0x12CC0233.4'/}','{uncore_cha_5/event=0x35,umask=0x21,config1=0x12CC023300000000,name='UNC_CHA_TOR_INSERTS.IA_MISS.0x12CC0233.5'/}','{uncore_cha_6/event=0x35,umask=0x21,config1=0x12CC023300000000,name='UNC_CHA_TOR_INSERTS.IA_MISS.0x12CC0233.6'/}','{uncore_cha_7/event=0x35,umask=0x21,config1=0x12CC023300000000,name='UNC_CHA_TOR_INSERTS.IA_MISS.0x12CC0233.7'/}','{uncore_cha_8/event=0x35,umask=0x21,config1=0x12CC023300000000,name='UNC_CHA_TOR_INSERTS.IA_MISS.0x12CC0233.8'/}','{uncore_cha_9/event=0x35,umask=0x21,config1=0x12CC023300000000,name='UNC_CHA_TOR_INSERTS.IA_MISS.0x12CC0233.9'/}','{uncore_cha_10/event=0x35,umask=0x21,config1=0x12CC023300000000,name='UNC_CHA_TOR_INSERTS.IA_MISS.0x12CC0233.10'/}','{uncore_cha_11/event=0x35,umask=0x21,config1=0x12CC023300000000,name='UNC_CHA_TOR_INSERTS.IA_MISS.0x12CC0233.11'/}','{uncore_cha_12/event=0x35,umask=0x21,config1=0x12CC023300000000,name='UNC_CHA_TOR_INSERTS.IA_MISS.0x12CC0233.12'/}','{uncore_cha_13/event=0x35,umask=0x21,config1=0x12CC023300000000,name='UNC_CHA_TOR_INSERTS.IA_MISS.0x12CC0233.13'/}','{uncore_cha_14/event=0x35,umask=0x21,config1=0x12CC023300000000,name='UNC_CHA_TOR_INSERTS.IA_MISS.0x12CC0233.14'/}','{uncore_cha_15/event=0x35,umask=0x21,config1=0x12CC023300000000,name='UNC_CHA_TOR_INSERTS.IA_MISS.0x12CC0233.15'/}','{uncore_cha_16/event=0x35,umask=0x21,config1=0x12CC023300000000,name='UNC_CHA_TOR_INSERTS.IA_MISS.0x12CC0233.16'/}','{uncore_cha_17/event=0x35,umask=0x21,config1=0x12CC023300000000,name='UNC_CHA_TOR_INSERTS.IA_MISS.0x12CC0233.17'/}','{uncore_cha_18/event=0x35,umask=0x21,config1=0x12CC023300000000,name='UNC_CHA_TOR_INSERTS.IA_MISS.0x12CC0233.18'/}','{uncore_cha_19/event=0x35,umask=0x21,config1=0x12CC023300000000,name='UNC_CHA_TOR_INSERTS.IA_MISS.0x12CC0233.19'/}','{uncore_cha_20/event=0x35,umask=0x21,config1=0x12CC023300000000,name='UNC_CHA_TOR_INSERTS.IA_MISS.0x12CC0233.20'/}','{uncore_cha_21/event=0x35,umask=0x21,config1=0x12CC023300000000,name='UNC_CHA_TOR_INSERTS.IA_MISS.0x12CC0233.21'/}','{uncore_cha_22/event=0x35,umask=0x21,config1=0x12CC023300000000,name='UNC_CHA_TOR_INSERTS.IA_MISS.0x12CC0233.22'/}','{uncore_cha_23/event=0x35,umask=0x21,config1=0x12CC023300000000,name='UNC_CHA_TOR_INSERTS.IA_MISS.0x12CC0233.23'/}','{uncore_cha_24/event=0x35,umask=0x21,config1=0x12CC023300000000,name='UNC_CHA_TOR_INSERTS.IA_MISS.0x12CC0233.24'/}','{uncore_cha_25/event=0x35,umask=0x21,config1=0x12CC023300000000,name='UNC_CHA_TOR_INSERTS.IA_MISS.0x12CC0233.25'/}','{uncore_cha_26/event=0x35,umask=0x21,config1=0x12CC023300000000,name='UNC_CHA_TOR_INSERTS.IA_MISS.0x12CC0233.26'/}','{uncore_cha_27/event=0x35,umask=0x21,config1=0x12CC023300000000,name='UNC_CHA_TOR_INSERTS.IA_MISS.0x12CC0233.27'/}','{uncore_cha_0/event=0x35,umask=0x21,config1=0x12D4043300000000,name='UNC_CHA_TOR_INSERTS.IA_MISS.0x12D40433.0'/}','{uncore_cha_1/event=0x35,umask=0x21,config1=0x12D4043300000000,name='UNC_CHA_TOR_INSERTS.IA_MISS.0x12D40433.1'/}','{uncore_cha_2/event=0x35,umask=0x21,config1=0x12D4043300000000,name='UNC_CHA_TOR_INSERTS.IA_MISS.0x12D40433.2'/}','{uncore_cha_3/event=0x35,umask=0x21,config1=0x12D4043300000000,name='UNC_CHA_TOR_INSERTS.IA_MISS.0x12D40433.3'/}','{uncore_cha_4/event=0x35,umask=0x21,config1=0x12D4043300000000,name='UNC_CHA_TOR_INSERTS.IA_MISS.0x12D40433.4'/}','{uncore_cha_5/event=0x35,umask=0x21,config1=0x12D4043300000000,name='UNC_CHA_TOR_INSERTS.IA_MISS.0x12D40433.5'/}','{uncore_cha_6/event=0x35,umask=0x21,config1=0x12D4043300000000,name='UNC_CHA_TOR_INSERTS.IA_MISS.0x12D40433.6'/}','{uncore_cha_7/event=0x35,umask=0x21,config1=0x12D4043300000000,name='UNC_CHA_TOR_INSERTS.IA_MISS.0x12D40433.7'/}','{uncore_cha_8/event=0x35,umask=0x21,config1=0x12D4043300000000,name='UNC_CHA_TOR_INSERTS.IA_MISS.0x12D40433.8'/}','{uncore_cha_9/event=0x35,umask=0x21,config1=0x12D4043300000000,name='UNC_CHA_TOR_INSERTS.IA_MISS.0x12D40433.9'/}','{uncore_cha_10/event=0x35,umask=0x21,config1=0x12D4043300000000,name='UNC_CHA_TOR_INSERTS.IA_MISS.0x12D40433.10'/}','{uncore_cha_11/event=0x35,umask=0x21,config1=0x12D4043300000000,name='UNC_CHA_TOR_INSERTS.IA_MISS.0x12D40433.11'/}','{uncore_cha_12/event=0x35,umask=0x21,config1=0x12D4043300000000,name='UNC_CHA_TOR_INSERTS.IA_MISS.0x12D40433.12'/}','{uncore_cha_13/event=0x35,umask=0x21,config1=0x12D4043300000000,name='UNC_CHA_TOR_INSERTS.IA_MISS.0x12D40433.13'/}','{uncore_cha_14/event=0x35,umask=0x21,config1=0x12D4043300000000,name='UNC_CHA_TOR_INSERTS.IA_MISS.0x12D40433.14'/}','{uncore_cha_15/event=0x35,umask=0x21,config1=0x12D4043300000000,name='UNC_CHA_TOR_INSERTS.IA_MISS.0x12D40433.15'/}','{uncore_cha_16/event=0x35,umask=0x21,config1=0x12D4043300000000,name='UNC_CHA_TOR_INSERTS.IA_MISS.0x12D40433.16'/}','{uncore_cha_17/event=0x35,umask=0x21,config1=0x12D4043300000000,name='UNC_CHA_TOR_INSERTS.IA_MISS.0x12D40433.17'/}','{uncore_cha_18/event=0x35,umask=0x21,config1=0x12D4043300000000,name='UNC_CHA_TOR_INSERTS.IA_MISS.0x12D40433.18'/}','{uncore_cha_19/event=0x35,umask=0x21,config1=0x12D4043300000000,name='UNC_CHA_TOR_INSERTS.IA_MISS.0x12D40433.19'/}','{uncore_cha_20/event=0x35,umask=0x21,config1=0x12D4043300000000,name='UNC_CHA_TOR_INSERTS.IA_MISS.0x12D40433.20'/}','{uncore_cha_21/event=0x35,umask=0x21,config1=0x12D4043300000000,name='UNC_CHA_TOR_INSERTS.IA_MISS.0x12D40433.21'/}','{uncore_cha_22/event=0x35,umask=0x21,config1=0x12D4043300000000,name='UNC_CHA_TOR_INSERTS.IA_MISS.0x12D40433.22'/}','{uncore_cha_23/event=0x35,umask=0x21,config1=0x12D4043300000000,name='UNC_CHA_TOR_INSERTS.IA_MISS.0x12D40433.23'/}','{uncore_cha_24/event=0x35,umask=0x21,config1=0x12D4043300000000,name='UNC_CHA_TOR_INSERTS.IA_MISS.0x12D40433.24'/}','{uncore_cha_25/event=0x35,umask=0x21,config1=0x12D4043300000000,name='UNC_CHA_TOR_INSERTS.IA_MISS.0x12D40433.25'/}','{uncore_cha_26/event=0x35,umask=0x21,config1=0x12D4043300000000,name='UNC_CHA_TOR_INSERTS.IA_MISS.0x12D40433.26'/}','{uncore_cha_27/event=0x35,umask=0x21,config1=0x12D4043300000000,name='UNC_CHA_TOR_INSERTS.IA_MISS.0x12D40433.27'/}','{uncore_cha_0/event=0x35,umask=0x21,config1=0x12C4003300000000,name='UNC_CHA_TOR_INSERTS.IA_MISS.0x12C40033.0'/}','{uncore_cha_1/event=0x35,umask=0x21,config1=0x12C4003300000000,name='UNC_CHA_TOR_INSERTS.IA_MISS.0x12C40033.1'/}','{uncore_cha_2/event=0x35,umask=0x21,config1=0x12C4003300000000,name='UNC_CHA_TOR_INSERTS.IA_MISS.0x12C40033.2'/}','{uncore_cha_3/event=0x35,umask=0x21,config1=0x12C4003300000000,name='UNC_CHA_TOR_INSERTS.IA_MISS.0x12C40033.3'/}','{uncore_cha_4/event=0x35,umask=0x21,config1=0x12C4003300000000,name='UNC_CHA_TOR_INSERTS.IA_MISS.0x12C40033.4'/}','{uncore_cha_5/event=0x35,umask=0x21,config1=0x12C4003300000000,name='UNC_CHA_TOR_INSERTS.IA_MISS.0x12C40033.5'/}','{uncore_cha_6/event=0x35,umask=0x21,config1=0x12C4003300000000,name='UNC_CHA_TOR_INSERTS.IA_MISS.0x12C40033.6'/}','{uncore_cha_7/event=0x35,umask=0x21,config1=0x12C4003300000000,name='UNC_CHA_TOR_INSERTS.IA_MISS.0x12C40033.7'/}','{uncore_cha_8/event=0x35,umask=0x21,config1=0x12C4003300000000,name='UNC_CHA_TOR_INSERTS.IA_MISS.0x12C40033.8'/}','{uncore_cha_9/event=0x35,umask=0x21,config1=0x12C4003300000000,name='UNC_CHA_TOR_INSERTS.IA_MISS.0x12C40033.9'/}','{uncore_cha_10/event=0x35,umask=0x21,config1=0x12C4003300000000,name='UNC_CHA_TOR_INSERTS.IA_MISS.0x12C40033.10'/}','{uncore_cha_11/event=0x35,umask=0x21,config1=0x12C4003300000000,name='UNC_CHA_TOR_INSERTS.IA_MISS.0x12C40033.11'/}','{uncore_cha_12/event=0x35,umask=0x21,config1=0x12C4003300000000,name='UNC_CHA_TOR_INSERTS.IA_MISS.0x12C40033.12'/}','{uncore_cha_13/event=0x35,umask=0x21,config1=0x12C4003300000000,name='UNC_CHA_TOR_INSERTS.IA_MISS.0x12C40033.13'/}','{uncore_cha_14/event=0x35,umask=0x21,config1=0x12C4003300000000,name='UNC_CHA_TOR_INSERTS.IA_MISS.0x12C40033.14'/}','{uncore_cha_15/event=0x35,umask=0x21,config1=0x12C4003300000000,name='UNC_CHA_TOR_INSERTS.IA_MISS.0x12C40033.15'/}','{uncore_cha_16/event=0x35,umask=0x21,config1=0x12C4003300000000,name='UNC_CHA_TOR_INSERTS.IA_MISS.0x12C40033.16'/}','{uncore_cha_17/event=0x35,umask=0x21,config1=0x12C4003300000000,name='UNC_CHA_TOR_INSERTS.IA_MISS.0x12C40033.17'/}','{uncore_cha_18/event=0x35,umask=0x21,config1=0x12C4003300000000,name='UNC_CHA_TOR_INSERTS.IA_MISS.0x12C40033.18'/}','{uncore_cha_19/event=0x35,umask=0x21,config1=0x12C4003300000000,name='UNC_CHA_TOR_INSERTS.IA_MISS.0x12C40033.19'/}','{uncore_cha_20/event=0x35,umask=0x21,config1=0x12C4003300000000,name='UNC_CHA_TOR_INSERTS.IA_MISS.0x12C40033.20'/}','{uncore_cha_21/event=0x35,umask=0x21,config1=0x12C4003300000000,name='UNC_CHA_TOR_INSERTS.IA_MISS.0x12C40033.21'/}','{uncore_cha_22/event=0x35,umask=0x21,config1=0x12C4003300000000,name='UNC_CHA_TOR_INSERTS.IA_MISS.0x12C40033.22'/}','{uncore_cha_23/event=0x35,umask=0x21,config1=0x12C4003300000000,name='UNC_CHA_TOR_INSERTS.IA_MISS.0x12C40033.23'/}','{uncore_cha_24/event=0x35,umask=0x21,config1=0x12C4003300000000,name='UNC_CHA_TOR_INSERTS.IA_MISS.0x12C40033.24'/}','{uncore_cha_25/event=0x35,umask=0x21,config1=0x12C4003300000000,name='UNC_CHA_TOR_INSERTS.IA_MISS.0x12C40033.25'/}','{uncore_cha_26/event=0x35,umask=0x21,config1=0x12C4003300000000,name='UNC_CHA_TOR_INSERTS.IA_MISS.0x12C40033.26'/}','{uncore_cha_27/event=0x35,umask=0x21,config1=0x12C4003300000000,name='UNC_CHA_TOR_INSERTS.IA_MISS.0x12C40033.27'/}','{uncore_iio_0/event=0x83,umask=0x04,ch_mask=0x00,fc_mask=0x07,name='UNC_IIO_DATA_REQ_OF_CPU.MEM_READ.PART0.0'/,uncore_iio_0/event=0x83,umask=0x04,ch_mask=0x02,fc_mask=0x07,name='UNC_IIO_DATA_REQ_OF_CPU.MEM_READ.PART1.0'/}','{uncore_iio_1/event=0x83,umask=0x04,ch_mask=0x00,fc_mask=0x07,name='UNC_IIO_DATA_REQ_OF_CPU.MEM_READ.PART0.1'/,uncore_iio_1/event=0x83,umask=0x04,ch_mask=0x02,fc_mask=0x07,name='UNC_IIO_DATA_REQ_OF_CPU.MEM_READ.PART1.1'/}','{uncore_iio_2/event=0x83,umask=0x04,ch_mask=0x00,fc_mask=0x07,name='UNC_IIO_DATA_REQ_OF_CPU.MEM_READ.PART0.2'/,uncore_iio_2/event=0x83,umask=0x04,ch_mask=0x02,fc_mask=0x07,name='UNC_IIO_DATA_REQ_OF_CPU.MEM_READ.PART1.2'/}','{uncore_iio_3/event=0x83,umask=0x04,ch_mask=0x00,fc_mask=0x07,name='UNC_IIO_DATA_REQ_OF_CPU.MEM_READ.PART0.3'/,uncore_iio_3/event=0x83,umask=0x04,ch_mask=0x02,fc_mask=0x07,name='UNC_IIO_DATA_REQ_OF_CPU.MEM_READ.PART1.3'/}','{uncore_iio_4/event=0x83,umask=0x04,ch_mask=0x00,fc_mask=0x07,name='UNC_IIO_DATA_REQ_OF_CPU.MEM_READ.PART0.4'/,uncore_iio_4/event=0x83,umask=0x04,ch_mask=0x02,fc_mask=0x07,name='UNC_IIO_DATA_REQ_OF_CPU.MEM_READ.PART1.4'/}','{uncore_iio_0/event=0x83,umask=0x04,ch_mask=0x04,fc_mask=0x07,name='UNC_IIO_DATA_REQ_OF_CPU.MEM_READ.PART2.0'/,uncore_iio_0/event=0x83,umask=0x04,ch_mask=0x08,fc_mask=0x07,name='UNC_IIO_DATA_REQ_OF_CPU.MEM_READ.PART3.0'/}','{uncore_iio_1/event=0x83,umask=0x04,ch_mask=0x04,fc_mask=0x07,name='UNC_IIO_DATA_REQ_OF_CPU.MEM_READ.PART2.1'/,uncore_iio_1/event=0x83,umask=0x04,ch_mask=0x08,fc_mask=0x07,name='UNC_IIO_DATA_REQ_OF_CPU.MEM_READ.PART3.1'/}','{uncore_iio_2/event=0x83,umask=0x04,ch_mask=0x04,fc_mask=0x07,name='UNC_IIO_DATA_REQ_OF_CPU.MEM_READ.PART2.2'/,uncore_iio_2/event=0x83,umask=0x04,ch_mask=0x08,fc_mask=0x07,name='UNC_IIO_DATA_REQ_OF_CPU.MEM_READ.PART3.2'/}','{uncore_iio_3/event=0x83,umask=0x04,ch_mask=0x04,fc_mask=0x07,name='UNC_IIO_DATA_REQ_OF_CPU.MEM_READ.PART2.3'/,uncore_iio_3/event=0x83,umask=0x04,ch_mask=0x08,fc_mask=0x07,name='UNC_IIO_DATA_REQ_OF_CPU.MEM_READ.PART3.3'/}','{uncore_iio_4/event=0x83,umask=0x04,ch_mask=0x04,fc_mask=0x07,name='UNC_IIO_DATA_REQ_OF_CPU.MEM_READ.PART2.4'/,uncore_iio_4/event=0x83,umask=0x04,ch_mask=0x08,fc_mask=0x07,name='UNC_IIO_DATA_REQ_OF_CPU.MEM_READ.PART3.4'/}','{uncore_iio_0/event=0x83,umask=0x01,ch_mask=0x00,fc_mask=0x07,name='UNC_IIO_DATA_REQ_OF_CPU.MEM_WRITE.PART0.0'/,uncore_iio_0/event=0x83,umask=0x01,ch_mask=0x02,fc_mask=0x07,name='UNC_IIO_DATA_REQ_OF_CPU.MEM_WRITE.PART1.0'/}','{uncore_iio_1/event=0x83,umask=0x01,ch_mask=0x00,fc_mask=0x07,name='UNC_IIO_DATA_REQ_OF_CPU.MEM_WRITE.PART0.1'/,uncore_iio_1/event=0x83,umask=0x01,ch_mask=0x02,fc_mask=0x07,name='UNC_IIO_DATA_REQ_OF_CPU.MEM_WRITE.PART1.1'/}','{uncore_iio_2/event=0x83,umask=0x01,ch_mask=0x00,fc_mask=0x07,name='UNC_IIO_DATA_REQ_OF_CPU.MEM_WRITE.PART0.2'/,uncore_iio_2/event=0x83,umask=0x01,ch_mask=0x02,fc_mask=0x07,name='UNC_IIO_DATA_REQ_OF_CPU.MEM_WRITE.PART1.2'/}','{uncore_iio_3/event=0x83,umask=0x01,ch_mask=0x00,fc_mask=0x07,name='UNC_IIO_DATA_REQ_OF_CPU.MEM_WRITE.PART0.3'/,uncore_iio_3/event=0x83,umask=0x01,ch_mask=0x02,fc_mask=0x07,name='UNC_IIO_DATA_REQ_OF_CPU.MEM_WRITE.PART1.3'/}','{uncore_iio_4/event=0x83,umask=0x01,ch_mask=0x00,fc_mask=0x07,name='UNC_IIO_DATA_REQ_OF_CPU.MEM_WRITE.PART0.4'/,uncore_iio_4/event=0x83,umask=0x01,ch_mask=0x02,fc_mask=0x07,name='UNC_IIO_DATA_REQ_OF_CPU.MEM_WRITE.PART1.4'/}','{uncore_iio_0/event=0x83,umask=0x01,ch_mask=0x04,fc_mask=0x07,name='UNC_IIO_DATA_REQ_OF_CPU.MEM_WRITE.PART2.0'/,uncore_iio_0/event=0x83,umask=0x01,ch_mask=0x08,fc_mask=0x07,name='UNC_IIO_DATA_REQ_OF_CPU.MEM_WRITE.PART3.0'/}','{uncore_iio_1/event=0x83,umask=0x01,ch_mask=0x04,fc_mask=0x07,name='UNC_IIO_DATA_REQ_OF_CPU.MEM_WRITE.PART2.1'/,uncore_iio_1/event=0x83,umask=0x01,ch_mask=0x08,fc_mask=0x07,name='UNC_IIO_DATA_REQ_OF_CPU.MEM_WRITE.PART3.1'/}','{uncore_iio_2/event=0x83,umask=0x01,ch_mask=0x04,fc_mask=0x07,name='UNC_IIO_DATA_REQ_OF_CPU.MEM_WRITE.PART2.2'/,uncore_iio_2/event=0x83,umask=0x01,ch_mask=0x08,fc_mask=0x07,name='UNC_IIO_DATA_REQ_OF_CPU.MEM_WRITE.PART3.2'/}','{uncore_iio_3/event=0x83,umask=0x01,ch_mask=0x04,fc_mask=0x07,name='UNC_IIO_DATA_REQ_OF_CPU.MEM_WRITE.PART2.3'/,uncore_iio_3/event=0x83,umask=0x01,ch_mask=0x08,fc_mask=0x07,name='UNC_IIO_DATA_REQ_OF_CPU.MEM_WRITE.PART3.3'/}','{uncore_iio_4/event=0x83,umask=0x01,ch_mask=0x04,fc_mask=0x07,name='UNC_IIO_DATA_REQ_OF_CPU.MEM_WRITE.PART2.4'/,uncore_iio_4/event=0x83,umask=0x01,ch_mask=0x08,fc_mask=0x07,name='UNC_IIO_DATA_REQ_OF_CPU.MEM_WRITE.PART3.4'/}','{uncore_upi_0/event=0x2,umask=0x0f,name='UNC_UPI_TxL_FLITS.ALL_DATA.0'/,uncore_upi_0/event=0x2,umask=0x97,name='UNC_UPI_TxL_FLITS.NON_DATA.0'/,uncore_upi_0/event=0x1,umask=0x0,name='UNC_UPI_CLOCKTICKS.0'/,uncore_upi_0/event=0x21,umask=0x0,name='UNC_UPI_L1_POWER_CYCLES.0'/}','{uncore_upi_1/event=0x2,umask=0x0f,name='UNC_UPI_TxL_FLITS.ALL_DATA.1'/,uncore_upi_1/event=0x2,umask=0x97,name='UNC_UPI_TxL_FLITS.NON_DATA.1'/,uncore_upi_1/event=0x1,umask=0x0,name='UNC_UPI_CLOCKTICKS.1'/,uncore_upi_1/event=0x21,umask=0x0,name='UNC_UPI_L1_POWER_CYCLES.1'/}','{uncore_upi_2/event=0x2,umask=0x0f,name='UNC_UPI_TxL_FLITS.ALL_DATA.2'/,uncore_upi_2/event=0x2,umask=0x97,name='UNC_UPI_TxL_FLITS.NON_DATA.2'/,uncore_upi_2/event=0x1,umask=0x0,name='UNC_UPI_CLOCKTICKS.2'/,uncore_upi_2/event=0x21,umask=0x0,name='UNC_UPI_L1_POWER_CYCLES.2'/}' sleep 10
2023-06-06 09:33:25,318 INFO: Collecting perf stat for events in : clx_skx.txt
2023-06-06 09:33:38,351 INFO: Collection complete! Calculating TSC frequency now
2023-06-06 09:33:39,398 INFO: perf stat dumped to perfstat.csv

./perf-postprocess -v -r perfstat.csv
2023-06-06 09:34:46,567 WARNING: MULTIPLE GROUPS: metric "metric_LLC MPI (includes code+data+rfo w/ prefetches)", events "{'UNC_CHA_TOR_INSERTS.IA_MISS.0x12D40433', 'instructions', 'UNC_CHA_TOR_INSERTS.IA_MISS.0x12CC0233', 'UNC_CHA_TOR_INSERTS.IA_MISS.0x12C40033'}"
2023-06-06 09:34:46,568 WARNING: MULTIPLE GROUPS: metric "metric_LLC code read MPI (demand+prefetch)", events "{'instructions', 'UNC_CHA_TOR_INSERTS.IA_MISS.0x12CC0233'}"
2023-06-06 09:34:46,568 WARNING: MULTIPLE GROUPS: metric "metric_LLC data read MPI (demand+prefetch)", events "{'UNC_CHA_TOR_INSERTS.IA_MISS.0x12D40433', 'instructions'}"
2023-06-06 09:34:46,569 WARNING: MULTIPLE GROUPS: metric "metric_LLC total HITM (per instr)", events "{'OCR.ALL_READS.L3_MISS.REMOTE_HITM', 'instructions'}"
2023-06-06 09:34:46,569 WARNING: MULTIPLE GROUPS: metric "metric_LLC total HIT clean line forwards (per instr)", events "{'instructions', 'OCR.ALL_READS.L3_MISS.REMOTE_HIT_FORWARD'}"
2023-06-06 09:34:46,570 WARNING: MULTIPLE GROUPS: metric "metric_Average LLC data read miss latency (in clks)", events "{'OFFCORE_REQUESTS.L3_MISS_DEMAND_DATA_RD', 'OFFCORE_REQUESTS_OUTSTANDING.L3_MISS_DEMAND_DATA_RD'}"
2023-06-06 09:34:46,571 WARNING: ZERO DIVISION: metric "metric_Average LLC data read miss latency (in ns)" expression "(1000000000 * [UNC_CHA_TOR_OCCUPANCY.IA_MISS.0x40433] / [UNC_CHA_TOR_INSERTS.IA_MISS.0x40433]) / ( [UNC_CHA_CLOCKTICKS] / ([CHAS_PER_SOCKET] * [SOCKET_COUNT]) )" values "(1000000000 * 1923.8025305341798 / 0.0) / ( 131988459930.64507 / (16 * 2) )"
2023-06-06 09:34:46,571 WARNING: MULTIPLE GROUPS: metric "metric_Average LLC data read miss latency for LOCAL requests (in ns)", events "{'UNC_CHA_TOR_INSERTS.IA_MISS.0x40432', 'UNC_CHA_TOR_OCCUPANCY.IA_MISS.0x40432', 'UNC_CHA_CLOCKTICKS'}"
2023-06-06 09:34:46,571 WARNING: ZERO DIVISION: metric "metric_Average LLC data read miss latency for LOCAL requests (in ns)" expression "(1000000000 * [UNC_CHA_TOR_OCCUPANCY.IA_MISS.0x40432] / [UNC_CHA_TOR_INSERTS.IA_MISS.0x40432]) / ( [UNC_CHA_CLOCKTICKS] / ([CHAS_PER_SOCKET] * [SOCKET_COUNT]) )" values "(1000000000 * 1008.5917001436954 / 0.0) / ( 131988459930.64507 / (16 * 2) )"
2023-06-06 09:34:46,572 WARNING: MULTIPLE GROUPS: metric "metric_Average LLC data read miss latency for REMOTE requests (in ns)", events "{'UNC_CHA_TOR_INSERTS.IA_MISS.0x40431', 'UNC_CHA_CLOCKTICKS', 'UNC_CHA_TOR_OCCUPANCY.IA_MISS.0x40431'}"
2023-06-06 09:34:46,572 WARNING: ZERO DIVISION: metric "metric_Average LLC data read miss latency for REMOTE requests (in ns)" expression "(1000000000 * [UNC_CHA_TOR_OCCUPANCY.IA_MISS.0x40431] / [UNC_CHA_TOR_INSERTS.IA_MISS.0x40431]) / ( [UNC_CHA_CLOCKTICKS] / ([CHAS_PER_SOCKET] * [SOCKET_COUNT]) )" values "(1000000000 * 1008.5917001436954 / 0.0) / ( 131988459930.64507 / (16 * 2) )"
2023-06-06 09:34:46,575 WARNING: MULTIPLE GROUPS: metric "metric_NUMA %_Reads addressed to local DRAM", events "{'UNC_CHA_TOR_INSERTS.IA_MISS.0x40431', 'UNC_CHA_TOR_INSERTS.IA_MISS.0x40432'}"
2023-06-06 09:34:46,575 WARNING: ZERO DIVISION: metric "metric_NUMA %_Reads addressed to local DRAM" expression "100 * [UNC_CHA_TOR_INSERTS.IA_MISS.0x40432] / ([UNC_CHA_TOR_INSERTS.IA_MISS.0x40432] + [UNC_CHA_TOR_INSERTS.IA_MISS.0x40431])" values "100 * 0.0 / (0.0 + 0.0)"
2023-06-06 09:34:46,575 WARNING: MULTIPLE GROUPS: metric "metric_NUMA %_Reads addressed to remote DRAM", events "{'UNC_CHA_TOR_INSERTS.IA_MISS.0x40431', 'UNC_CHA_TOR_INSERTS.IA_MISS.0x40432'}"
2023-06-06 09:34:46,576 WARNING: ZERO DIVISION: metric "metric_NUMA %_Reads addressed to remote DRAM" expression "100 * [UNC_CHA_TOR_INSERTS.IA_MISS.0x40431] / ([UNC_CHA_TOR_INSERTS.IA_MISS.0x40432] + [UNC_CHA_TOR_INSERTS.IA_MISS.0x40431])" values "100 * 0.0 / (0.0 + 0.0)"
2023-06-06 09:34:46,577 WARNING: MULTIPLE GROUPS: metric "metric_UPI Transmit utilization_% (includes control)", events "{'UNC_UPI_L1_POWER_CYCLES', 'cstate_pkg/c6-residency/', 'UNC_UPI_CLOCKTICKS', 'UNC_UPI_TxL_FLITS.ALL_DATA', 'UNC_UPI_TxL_FLITS.NON_DATA'}"
2023-06-06 09:34:46,577 WARNING: MISSING EVENTS: metric "metric_UPI Transmit utilization_% (includes control)" events "['cstate_pkg/c6-residency/']"
2023-06-06 09:34:46,577 WARNING: MISSING EVENTS: metric "metric_core c6 residency %" events "['cstate_core/c6-residency/']"
2023-06-06 09:34:46,578 WARNING: MISSING EVENTS: metric "metric_package c6 residency %" events "['cstate_pkg/c6-residency/']"
2023-06-06 09:34:46,581 WARNING: MULTIPLE GROUPS: metric "metric_IO_bandwidth_disk_or_network_writes (MB/sec)", events "{'UNC_IIO_DATA_REQ_OF_CPU.MEM_READ.PART3', 'UNC_IIO_DATA_REQ_OF_CPU.MEM_READ.PART2', 'UNC_IIO_DATA_REQ_OF_CPU.MEM_READ.PART0', 'UNC_IIO_DATA_REQ_OF_CPU.MEM_READ.PART1'}"
2023-06-06 09:34:46,581 WARNING: MULTIPLE GROUPS: metric "metric_IO_bandwidth_disk_or_network_reads (MB/sec)", events "{'UNC_IIO_DATA_REQ_OF_CPU.MEM_WRITE.PART0', 'UNC_IIO_DATA_REQ_OF_CPU.MEM_WRITE.PART1', 'UNC_IIO_DATA_REQ_OF_CPU.MEM_WRITE.PART3', 'UNC_IIO_DATA_REQ_OF_CPU.MEM_WRITE.PART2'}"
2023-06-06 09:34:46,583 WARNING: MULTIPLE GROUPS: metric "metric_TMA_..Frontend_Latency(%)", events "{'IDQ_UOPS_NOT_DELIVERED.CYCLES_0_UOPS_DELIV.CORE', 'CPU_CLK_UNHALTED.THREAD_ANY'}"
2023-06-06 09:34:46,584 WARNING: MULTIPLE GROUPS: metric "metric_TMA_....Branch_Resteers(%)", events "{'cpu-cycles', 'INT_MISC.CLEAR_RESTEER_CYCLES', 'BACLEARS.ANY'}"
2023-06-06 09:34:46,586 WARNING: MULTIPLE GROUPS: metric "metric_TMA_..Frontend_Bandwidth(%)", events "{'IDQ_UOPS_NOT_DELIVERED.CORE', 'IDQ_UOPS_NOT_DELIVERED.CYCLES_0_UOPS_DELIV.CORE', 'CPU_CLK_UNHALTED.THREAD_ANY'}"
2023-06-06 09:34:46,587 WARNING: MULTIPLE GROUPS: metric "metric_TMA_....MITE(%)", events "{'cpu-cycles', 'IDQ.ALL_MITE_CYCLES_4_UOPS', 'IDQ.ALL_MITE_CYCLES_ANY_UOPS', 'CPU_CLK_UNHALTED.THREAD_ANY'}"
2023-06-06 09:34:46,587 WARNING: MULTIPLE GROUPS: metric "metric_TMA_....DSB(%)", events "{'IDQ.ALL_DSB_CYCLES_4_UOPS', 'cpu-cycles', 'CPU_CLK_UNHALTED.THREAD_ANY', 'IDQ.ALL_DSB_CYCLES_ANY_UOPS'}"
2023-06-06 09:34:46,588 WARNING: MULTIPLE GROUPS: metric "metric_TMA_Bad_Speculation(%)", events "{'CPU_CLK_UNHALTED.THREAD_ANY', 'UOPS_RETIRED.RETIRE_SLOTS', 'INT_MISC.RECOVERY_CYCLES', 'cpu-cycles', 'UOPS_ISSUED.ANY', 'INT_MISC.RECOVERY_CYCLES_ANY'}"
2023-06-06 09:34:46,589 WARNING: MULTIPLE GROUPS: metric "metric_TMA_..Branch_Mispredicts(%)", events "{'UOPS_RETIRED.RETIRE_SLOTS', 'MACHINE_CLEARS.COUNT', 'CPU_CLK_UNHALTED.THREAD_ANY', 'BR_MISP_RETIRED.ALL_BRANCHES', 'INT_MISC.RECOVERY_CYCLES', 'cpu-cycles', 'UOPS_ISSUED.ANY', 'INT_MISC.RECOVERY_CYCLES_ANY'}"
2023-06-06 09:34:46,591 WARNING: MULTIPLE GROUPS: metric "metric_TMA_..Machine_Clears(%)", events "{'CPU_CLK_UNHALTED.THREAD_ANY', 'UOPS_RETIRED.RETIRE_SLOTS', 'MACHINE_CLEARS.COUNT', 'BR_MISP_RETIRED.ALL_BRANCHES', 'INT_MISC.RECOVERY_CYCLES', 'cpu-cycles', 'UOPS_ISSUED.ANY', 'INT_MISC.RECOVERY_CYCLES_ANY'}"
2023-06-06 09:34:46,592 WARNING: MULTIPLE GROUPS: metric "metric_TMA_Backend_Bound(%)", events "{'IDQ_UOPS_NOT_DELIVERED.CORE', 'CPU_CLK_UNHALTED.THREAD_ANY', 'INT_MISC.RECOVERY_CYCLES', 'cpu-cycles', 'UOPS_ISSUED.ANY', 'INT_MISC.RECOVERY_CYCLES_ANY'}"
2023-06-06 09:34:46,594 WARNING: MULTIPLE GROUPS: metric "metric_TMA_..Memory_Bound(%)", events "{'IDQ_UOPS_NOT_DELIVERED.CORE', 'UOPS_RETIRED.RETIRE_SLOTS', 'CPU_CLK_UNHALTED.THREAD_ANY', 'EXE_ACTIVITY.BOUND_ON_STORES', 'CYCLE_ACTIVITY.STALLS_TOTAL', 'INT_MISC.RECOVERY_CYCLES', 'INT_MISC.RECOVERY_CYCLES_ANY', 'cpu-cycles', 'CYCLE_ACTIVITY.STALLS_MEM_ANY', 'UOPS_ISSUED.ANY', 'EXE_ACTIVITY.2_PORTS_UTIL', 'EXE_ACTIVITY.1_PORTS_UTIL'}"
2023-06-06 09:34:46,595 WARNING: MULTIPLE GROUPS: metric "metric_TMA_....L1_Bound(%)", events "{'cpu-cycles', 'CYCLE_ACTIVITY.STALLS_MEM_ANY', 'CYCLE_ACTIVITY.STALLS_L1D_MISS'}"
2023-06-06 09:34:46,596 WARNING: MULTIPLE GROUPS: metric "metric_TMA_......DTLB_Load(%)", events "{'CYCLE_ACTIVITY.CYCLES_L1D_MISS', 'DTLB_LOAD_MISSES.STLB_HIT:c1', 'cpu-cycles', 'CYCLE_ACTIVITY.CYCLES_MEM_ANY', 'DTLB_LOAD_MISSES.WALK_ACTIVE'}"
2023-06-06 09:34:46,597 WARNING: MULTIPLE GROUPS: metric "metric_TMA_......Lock_Latency(%)", events "{'L2_RQSTS.RFO_HIT', 'L2_RQSTS.ALL_RFO', 'MEM_INST_RETIRED.LOCK_LOADS', 'cpu-cycles', 'OFFCORE_REQUESTS_OUTSTANDING.CYCLES_WITH_DEMAND_RFO', 'MEM_INST_RETIRED.ALL_STORES'}"
2023-06-06 09:34:46,598 WARNING: MULTIPLE GROUPS: metric "metric_TMA_....L2_Bound(%)", events "{'MEM_LOAD_RETIRED.FB_HIT', 'CYCLE_ACTIVITY.STALLS_L2_MISS', 'MEM_LOAD_RETIRED.L1_MISS', 'cpu-cycles', 'CYCLE_ACTIVITY.STALLS_L1D_MISS', 'L1D_PEND_MISS.FB_FULL:c1', 'MEM_LOAD_RETIRED.L2_HIT'}"
2023-06-06 09:34:46,600 WARNING: MULTIPLE GROUPS: metric "metric_TMA_......Data_Sharing(%)", events "{'MEM_LOAD_L3_HIT_RETIRED.XSNP_HIT', 'MEM_LOAD_RETIRED.FB_HIT', 'ref-cycles', 'MEM_LOAD_L3_HIT_RETIRED.XSNP_HITM', 'OCR.DEMAND_DATA_RD.L3_HIT.HIT_OTHER_CORE_FWD', 'MEM_LOAD_RETIRED.L1_MISS', 'cpu-cycles', 'OCR.DEMAND_DATA_RD.L3_HIT.HITM_OTHER_CORE'}"
2023-06-06 09:34:46,603 WARNING: MULTIPLE GROUPS: metric "metric_TMA_......False_Sharing(%)", events "{'ref-cycles', 'OCR.PF_L2_RFO.L3_HIT.HITM_OTHER_CORE', 'cpu-cycles', 'OCR.DEMAND_RFO.L3_MISS.REMOTE_HITM', 'OCR.PF_L2_RFO.L3_MISS.REMOTE_HITM', 'OCR.DEMAND_RFO.L3_HIT.HITM_OTHER_CORE'}"
2023-06-06 09:34:46,605 WARNING: MULTIPLE GROUPS: metric "metric_TMA_..Core_Bound(%)", events "{'IDQ_UOPS_NOT_DELIVERED.CORE', 'CPU_CLK_UNHALTED.THREAD_ANY', 'UOPS_RETIRED.RETIRE_SLOTS', 'EXE_ACTIVITY.BOUND_ON_STORES', 'CYCLE_ACTIVITY.STALLS_TOTAL', 'INT_MISC.RECOVERY_CYCLES', 'EXE_ACTIVITY.1_PORTS_UTIL', 'cpu-cycles', 'CYCLE_ACTIVITY.STALLS_MEM_ANY', 'UOPS_ISSUED.ANY', 'EXE_ACTIVITY.2_PORTS_UTIL', 'INT_MISC.RECOVERY_CYCLES_ANY'}"
2023-06-06 09:34:46,607 WARNING: MULTIPLE GROUPS: metric "metric_TMA_....Ports_Utilization(%)", events "{'UOPS_RETIRED.RETIRE_SLOTS', 'CPU_CLK_UNHALTED.THREAD_ANY', 'EXE_ACTIVITY.EXE_BOUND_0_PORTS', 'ARITH.DIVIDER_ACTIVE', 'CYCLE_ACTIVITY.STALLS_TOTAL', 'cpu-cycles', 'CYCLE_ACTIVITY.STALLS_MEM_ANY', 'EXE_ACTIVITY.2_PORTS_UTIL', 'EXE_ACTIVITY.1_PORTS_UTIL'}"
2023-06-06 09:34:46,609 WARNING: MULTIPLE GROUPS: metric "metric_TMA_......0_Port_Utilized(%)", events "{'UOPS_EXECUTED.CORE_CYCLES_NONE', 'CPU_CLK_UNHALTED.THREAD_ANY', 'EXE_ACTIVITY.EXE_BOUND_0_PORTS'}"
2023-06-06 09:34:46,609 WARNING: MULTIPLE GROUPS: metric "metric_TMA_......1_Port_Utilized(%)", events "{'UOPS_EXECUTED.CORE_CYCLES_GE_1', 'UOPS_EXECUTED.CORE_CYCLES_GE_2', 'EXE_ACTIVITY.1_PORTS_UTIL', 'CPU_CLK_UNHALTED.THREAD_ANY'}"
2023-06-06 09:34:46,610 WARNING: MULTIPLE GROUPS: metric "metric_TMA_......2_Port_Utilized(%)", events "{'UOPS_EXECUTED.CORE_CYCLES_GE_3', 'UOPS_EXECUTED.CORE_CYCLES_GE_2', 'CPU_CLK_UNHALTED.THREAD_ANY', 'EXE_ACTIVITY.2_PORTS_UTIL'}"
2023-06-06 09:34:46,611 WARNING: MULTIPLE GROUPS: metric "metric_TMA_......3m_Ports_Utilized(%)", events "{'UOPS_EXECUTED.CORE_CYCLES_GE_3', 'CPU_CLK_UNHALTED.THREAD_ANY'}"
2023-06-06 09:34:46,611 WARNING: MULTIPLE GROUPS: metric "metric_TMA_Retiring(%)", events "{'cpu-cycles', 'UOPS_RETIRED.RETIRE_SLOTS', 'CPU_CLK_UNHALTED.THREAD_ANY'}"
2023-06-06 09:34:46,612 WARNING: MULTIPLE GROUPS: metric "metric_TMA_..Light_Operations(%)", events "{'UOPS_RETIRED.MACRO_FUSED', 'instructions', 'UOPS_RETIRED.RETIRE_SLOTS', 'CPU_CLK_UNHALTED.THREAD_ANY', 'cpu-cycles'}"
2023-06-06 09:34:46,613 WARNING: MULTIPLE GROUPS: metric "metric_TMA_....FP_Arith(%)", events "{'UOPS_RETIRED.RETIRE_SLOTS', 'CPU_CLK_UNHALTED.THREAD_ANY', 'UOPS_EXECUTED.X87', 'cpu-cycles', 'UOPS_EXECUTED.THREAD', 'FP_ARITH_INST_RETIRED.128B_PACKED_DOUBLE:u0xfc', 'FP_ARITH_INST_RETIRED.SCALAR_SINGLE:u0x03'}"
2023-06-06 09:34:46,614 WARNING: MULTIPLE GROUPS: metric "metric_TMA_......FP_Scalar(%)", events "{'UOPS_RETIRED.RETIRE_SLOTS', 'FP_ARITH_INST_RETIRED.SCALAR_SINGLE:u0x03'}"
2023-06-06 09:34:46,614 WARNING: MULTIPLE GROUPS: metric "metric_TMA_......FP_Vector(%)", events "{'UOPS_RETIRED.RETIRE_SLOTS', 'FP_ARITH_INST_RETIRED.128B_PACKED_DOUBLE:u0xfc'}"
2023-06-06 09:34:46,615 WARNING: MULTIPLE GROUPS: metric "metric_TMA_..Heavy_Operations(%)", events "{'UOPS_RETIRED.MACRO_FUSED', 'instructions', 'UOPS_RETIRED.RETIRE_SLOTS', 'CPU_CLK_UNHALTED.THREAD_ANY', 'cpu-cycles'}"
2023-06-06 09:34:46,616 WARNING: MULTIPLE GROUPS: metric "metric_TMA_..Microcode_Sequencer(%)", events "{'UOPS_ISSUED.ANY', 'IDQ.MS_UOPS', 'UOPS_RETIRED.RETIRE_SLOTS', 'CPU_CLK_UNHALTED.THREAD_ANY'}"
2023-06-06 09:34:46,616 WARNING: throwing out last sample because it was too short
2023-06-06 09:34:46,617 WARNING: 0 MISSING DATA: set()
2023-06-06 09:34:46,617 WARNING: 5 ZERO DIVISION: {'metric_Average LLC data read miss latency for LOCAL requests (in ns)', 'metric_Average LLC data read miss latency for REMOTE requests (in ns)', 'metric_NUMA %_Reads addressed to remote DRAM', 'metric_NUMA %_Reads addressed to local DRAM', 'metric_Average LLC data read miss latency (in ns)'}
2023-06-06 09:34:46,617 WARNING: 3 MISSING EVENTS: {'metric_UPI Transmit utilization_% (includes control)', 'metric_core c6 residency %', 'metric_package c6 residency %'}
2023-06-06 09:34:46,617 WARNING: 42 MULTIPLE GROUPS: {'metric_TMA_....Branch_Resteers(%)', 'metric_TMA_..Frontend_Latency(%)', 'metric_TMA_Backend_Bound(%)', 'metric_LLC total HIT clean line forwards (per instr)', 'metric_TMA_Retiring(%)', 'metric_IO_bandwidth_disk_or_network_writes (MB/sec)', 'metric_TMA_Bad_Speculation(%)', 'metric_TMA_......DTLB_Load(%)', 'metric_LLC data read MPI (demand+prefetch)', 'metric_TMA_....L1_Bound(%)', 'metric_Average LLC data read miss latency for LOCAL requests (in ns)', 'metric_UPI Transmit utilization_% (includes control)', 'metric_TMA_......1_Port_Utilized(%)', 'metric_TMA_....L2_Bound(%)', 'metric_NUMA %_Reads addressed to remote DRAM', 'metric_TMA_..Core_Bound(%)', 'metric_TMA_......3m_Ports_Utilized(%)', 'metric_LLC code read MPI (demand+prefetch)', 'metric_NUMA %_Reads addressed to local DRAM', 'metric_TMA_......2_Port_Utilized(%)', 'metric_TMA_......FP_Scalar(%)', 'metric_TMA_..Memory_Bound(%)', 'metric_LLC total HITM (per instr)', 'metric_TMA_..Light_Operations(%)', 'metric_Average LLC data read miss latency for REMOTE requests (in ns)', 'metric_Average LLC data read miss latency (in clks)', 'metric_TMA_..Microcode_Sequencer(%)', 'metric_TMA_......0_Port_Utilized(%)', 'metric_TMA_..Frontend_Bandwidth(%)', 'metric_TMA_......FP_Vector(%)', 'metric_LLC MPI (includes code+data+rfo w/ prefetches)', 'metric_TMA_....MITE(%)', 'metric_TMA_....Ports_Utilization(%)', 'metric_TMA_....FP_Arith(%)', 'metric_TMA_......Data_Sharing(%)', 'metric_TMA_....DSB(%)', 'metric_TMA_..Branch_Mispredicts(%)', 'metric_TMA_......False_Sharing(%)', 'metric_TMA_......Lock_Latency(%)', 'metric_TMA_..Machine_Clears(%)', 'metric_IO_bandwidth_disk_or_network_reads (MB/sec)', 'metric_TMA_..Heavy_Operations(%)'}
2023-06-06 09:34:46,639 INFO: Generated results file(s) in: /home/rjx/perfspect-1.2.11
2023-06-06 09:34:46,639 INFO: Done!

ls /sys/devices/
breakpoint  LNXSYSTM:00  pci0000:80  pnp0        uncore_cha_0   uncore_cha_13  uncore_cha_18  uncore_cha_22  uncore_cha_27  uncore_cha_7  uncore_iio_2  uncore_imc_2  uncore_irp_1  uncore_m2m_1    uncore_ubox
cpu         pci0000:00   pci0000:85  power       uncore_cha_1   uncore_cha_14  uncore_cha_19  uncore_cha_23  uncore_cha_3   uncore_cha_8  uncore_iio_3  uncore_imc_3  uncore_irp_2  uncore_m3upi_0  uncore_upi_0
intel_bts   pci0000:17   pci0000:ae  software    uncore_cha_10  uncore_cha_15  uncore_cha_2   uncore_cha_24  uncore_cha_4   uncore_cha_9  uncore_iio_4  uncore_imc_4  uncore_irp_3  uncore_m3upi_1  uncore_upi_1
intel_cqm   pci0000:3a   pci0000:d7  system      uncore_cha_11  uncore_cha_16  uncore_cha_20  uncore_cha_25  uncore_cha_5   uncore_iio_0  uncore_imc_0  uncore_imc_5  uncore_irp_4  uncore_m3upi_2  uncore_upi_2
intel_pt    pci0000:5d   platform    tracepoint  uncore_cha_12  uncore_cha_17  uncore_cha_21  uncore_cha_26  uncore_cha_6   uncore_iio_1  uncore_imc_1  uncore_irp_0  uncore_m2m_0  uncore_pcu      virtual
The results of the following indicators are also zero, and I'm not sure if it's for the same reason as the previous issue. Please help to investigate together. Thank you. metric avg
metric_LLC MPI (includes code+data+rfo w/ prefetches) 0
metric_LLC code read MPI (demand+prefetch) 0
metric_LLC data read MPI (demand+prefetch) 0
metric_Average LLC data read miss latency (in ns) 0
metric_Average LLC data read miss latency for LOCAL requests (in ns) 0
metric_Average LLC data read miss latency for REMOTE requests (in ns) 0
hilldani commented 1 year ago

so the following metrics aren't calculated because your system doesn't support cstate events:

  1. metricUPI Transmit utilization% (includes control)
  2. metric_core c6 residency %
  3. metric_package c6 residency %

The rest of the metrics which are zero are because of a ZERO DIVISION error because all your UNC_CHA_TOR_INSERTS.IA_MISS events are zero. We just need to figure out why.

hilldani commented 1 year ago

inside of your perfstat.csv file search for "UNC_CHA_TOR_INSERTS.IA_MISS". Does it show <not counted>, <not supported>, or 0? Also kernel version, perf version, and distro are you on (or upload your perfstat.csv file)? I am not able to reproduce this

jasonneverstop commented 1 year ago

Sorry, I'm not sure if it's allowed to upload perfstat.csv file. I'll try my best to provide you with the information you need.

In my perfstat.csv,the values of "UNC_CHA_TOR_INSERTS.IA_MISS" are all 0. image

The environment information is like this:

# perf -v
perf version 3.10.0-1160.90.1.el7.x86_64.debug

# uname -r
3.10.0-693.el7.x86_64

# cat /etc/centos-release
CentOS Linux release 7.9.2009 (Core)

# cat /etc/os-release
NAME="CentOS Linux"
VERSION="7 (Core)"
ID="centos"
ID_LIKE="rhel fedora"
VERSION_ID="7"
PRETTY_NAME="CentOS Linux 7 (Core)"
ANSI_COLOR="0;31"
CPE_NAME="cpe:/o:centos:centos:7"
HOME_URL="https://www.centos.org/"
BUG_REPORT_URL="https://bugs.centos.org/"

CENTOS_MANTISBT_PROJECT="CentOS-7"
CENTOS_MANTISBT_PROJECT_VERSION="7"
REDHAT_SUPPORT_PRODUCT="centos"
REDHAT_SUPPORT_PRODUCT_VERSION="7"
hilldani commented 1 year ago
thanks for sharing! This looks like some suspiciously idle DRAM. When I collect perfspect during a workload like sudo ./perf-collect -v -a "stress-ng --cpu 0 --cpu-load 70 --timeout 10s" I get values like this: value event
121654771 UNC_CHA_TOR_OCCUPANCY.IA_MISS.0x40432.0
590030 UNC_CHA_TOR_INSERTS.IA_MISS.0x40432.1

so even having 600 for UNC_CHA_TOR_OCCUPANCY is crazy low. What we can do to validate this is:

  1. Download memory latency checker https://www.intel.com/content/www/us/en/developer/articles/tool/intelr-memory-latency-checker.html
  2. Run it like this ./mlc --loaded_latency -d1000 -t5. This will:
    1. run a steady load for 5 seconds to find the DRAM bandwidth
    2. run for 100s at a steady state DRAM bandwidth
  3. collect Perfspect for 30s in between where bandwidth is known.

Please share the outcome of this

jasonneverstop commented 1 year ago

thanks for sharing! This looks like some suspiciously idle DRAM. When I collect perfspect during a workload like sudo ./perf-collect -v -a "stress-ng --cpu 0 --cpu-load 70 --timeout 10s" I get values like this:

value event 121654771 UNC_CHA_TOR_OCCUPANCY.IA_MISS.0x40432.0 590030 UNC_CHA_TOR_INSERTS.IA_MISS.0x40432.1 so even having 600 for UNC_CHA_TOR_OCCUPANCY is crazy low. What we can do to validate this is:

  1. Download memory latency checker https://www.intel.com/content/www/us/en/developer/articles/tool/intelr-memory-latency-checker.html

  2. Run it like this ./mlc --loaded_latency -d1000 -t5. This will:

    1. run a steady load for 5 seconds to find the DRAM bandwidth
    2. run for 100s at a steady state DRAM bandwidth
  3. collect Perfspect for 30s in between where bandwidth is known.

Please share the outcome of this

I don't quite understand how to use mlc. I ran it following the steps below. If it is incorrect, please let me know.

#./mlc --loaded_latency -d1000 -t5
Intel(R) Memory Latency Checker - v3.10
Command line parameters: --loaded_latency -d1000 -t5

Using buffer size of 100.000MiB/thread for reads and an additional 100.000MiB/thread for writes

Measuring Loaded Latencies for the system
Using all the threads from each core if Hyper-threading is enabled
Using Read-only traffic type
Inject  Latency Bandwidth
Delay   (ns)    MB/sec
==========================
 01000   89.89    22139.3

# ./mlc --loaded_latency -d1000 -t100  &  # ./perf-collect -t 30 -v
Intel(R) Memory Latency Checker - v3.10
Command line parameters: --loaded_latency -d1000 -t100

Using buffer size of 100.000MiB/thread for reads and an additional 100.000MiB/thread for writes

Measuring Loaded Latencies for the system
Using all the threads from each core if Hyper-threading is enabled
Using Read-only traffic type
Inject  Latency Bandwidth
Delay   (ns)    MB/sec
==========================
 01000   90.03    22107.7

image

hilldani commented 1 year ago

Hey we did successfully get a 3 in UNC_CHA_TOR_INSERTS.IA_MISS.0x40432.2, so your DIMM's are there. It might useful to see your DIMM configuration. Could you run https://github.com/intel/svr-info with

./svr-info -benchmark all -profile all -profile_duration 5 -profile_interval 1 -analyze all -analyze_duration 5 -analyze_frequency 5

if you don't want to post the results here you can also email it to me at daniel.hill@intel.com but my guess is there is something off with this server's setup.

hsane2001 commented 1 year ago

@jasonneverstop - When you run MLC the second time, you need to collect PerfSpect and look at the UNC events for memory bandwdith rather than the CHA events (which could be a separate issue). Pease post the values for below when running MLC. Expectation is to hit the 22GB/s value seen by MLC. imc/event=0x04,umask=0x03,name='UNC_M_CAS_COUNT.RD'/, imc/event=0x04,umask=0x0c,name='UNC_M_CAS_COUNT.WR'/;

hilldani commented 1 year ago

FYI This will be the metric "metric_memory bandwidth read (MB/sec)". From your idle system perfstat.csv these metrics seem to be working correctly.

image

Sending the updated perfstat during MLC will help validate further. In the meantime I'll try to setup a system with your exact kernel to check if TOR counters are up to date

hilldani commented 1 year ago

@jasonneverstop thanks for sending the files.

Measuring Loaded Latencies for the system
Using all the threads from each core if Hyper-threading is enabled
Using Read-only traffic type
Inject  Latency Bandwidth
Delay   (ns)    MB/sec
==========================
 01000   90.03    22107.7

DRAM read and write metrics work and are accurate

image

I'm still not able to reproduce the TOR event issues, my guess is there's some kernel support issue or something. I'll let you know if I find anything

jasonneverstop commented 1 year ago

Sure, thank you. I'll wait for your message.

hilldani commented 1 year ago

I have compared our system setups and the only thing that stands out to me is kernel version

jasonneverstop hilldani
3.10.0-693.el7.x86_64 3.10.0-1160.76.1.el7.x86_64

My guess is somewhere between these versions centos backported support for cha_tor events.

You can still calculate remote DRAM % without using cha_tor events

$\frac{\text{metric core initiated remote dram read bandwidth (MB/sec)}}{\text{metric memory bandwidth read (MB/sec)}}*100=\text{percent remote DRAM}$

As a sanity check we can validate that UPI BW (205 MB/s) is close to remote DRAM BW (171 MB/s). Of course, UPI BW will always be larger than remote DRAM BW because we are only measuring core-initiated reads.