search

RRZE-HPC / likwid

Performance monitoring and benchmarking suite
https://hpc.fau.de/research/tools/likwid/
GNU General Public License v3.0
1.69k stars 230 forks source link

Memory Bandwidth Numbers Reporting 0s for Zen2 Architecture #294

Open talnish opened 4 years ago

talnish commented 4 years ago

I am trying to profile my application on AMD Ryzen Threadripper 3960X (Zen2 arch). Specifically, I am trying to measure memory bandwidth and I get all 0's in the output. The OS I am using is Ubuntu 18.04. The command I used is the following: likwid-perfctr -g MEM -f [...app-arg...] The output looks something like this:

CPU name:   AMD Ryzen Threadripper 3960X 24-Core Processor 
CPU type:   AMD K17 (Zen2) architecture
CPU clock:  3.80 GHz

--------------------------------------------------------------------------------
Group 1: MEM
+------------------------------+---------+------------+
|             Event            | Counter |   Core 0   |
+------------------------------+---------+------------+
|       ACTUAL_CPU_CLOCK       |  FIXC1  | 8815591471 |
|     RETIRED_INSTRUCTIONS     |   PMC0  | 3983962068 |
|      CPU_CLOCKS_UNHALTED     |   PMC1  | 7166245156 |
| DATA_FROM_LOCAL_DRAM_CHANNEL |   DFC0  |          0 |
|  DATA_TO_LOCAL_DRAM_CHANNEL  |   DFC1  |          0 |
+------------------------------+---------+------------+

+-----------------------------------+---------+--------------+------------+--------------+--------------+
|               Event               | Counter |      Sum     |     Min    |      Max     |      Avg     |
+-----------------------------------+---------+--------------+------------+--------------+--------------+
|       ACTUAL_CPU_CLOCK STAT       |  FIXC1  | 484432371346 | 7098589982 |  77476122009 | 1.009234e+10 |
|         MAX_CPU_CLOCK STAT        |  FIXC2  | 435029922756 | 6428419720 |  67369824332 | 9.063123e+09 |
|     RETIRED_INSTRUCTIONS STAT     |   PMC0  | 359003536750 | 1954608551 | 187259446057 | 7.479240e+09 |
|      CPU_CLOCKS_UNHALTED STAT     |   PMC1  | 412355303377 | 5565474472 |  72902204537 | 8.590735e+09 |
| DATA_FROM_LOCAL_DRAM_CHANNEL STAT |   DFC0  |            0 |          0 |            0 |            0 |
|  DATA_TO_LOCAL_DRAM_CHANNEL STAT  |   DFC1  |            0 |          0 |            0 |            0 |
+-----------------------------------+---------+--------------+------------+--------------+--------------+

+-----------------------------------+-----------+
|               Metric              |   Core 0  |
+-----------------------------------+-----------+
|        Runtime (RDTSC) [s]        |   18.3399 |
|        Runtime unhalted [s]       |    2.3200 |
|            Clock [MHz]            | 4206.1307 |
|                CPI                |    1.7988 |
|  Memory read bandwidth [MBytes/s] |         0 |
|  Memory read data volume [GBytes] |         0 |
| Memory write bandwidth [MBytes/s] |         0 |
| Memory write data volume [GBytes] |         0 |
|    Memory data volume [GBytes]    |         0 |
+-----------------------------------+-----------+

+----------------------------------------+-------------+-----------+-----------+-----------+
|                 Metric                 |     Sum     |    Min    |    Max    |    Avg    |
+----------------------------------------+-------------+-----------+-----------+-----------+
|        Runtime (RDTSC) [s] STAT        |    880.3152 |   18.3399 |   18.3399 |   18.3399 |
|        Runtime unhalted [s] STAT       |    127.4866 |    1.8681 |   20.3892 |    2.6560 |
|            Clock [MHz] STAT            | 202028.1228 | 4043.6894 | 4369.8945 | 4208.9192 |
|                CPI STAT                |     95.3508 |    0.3893 |    2.9218 |    1.9865 |
|  Memory read bandwidth [MBytes/s] STAT |           0 |         0 |         0 |         0 |
|  Memory read data volume [GBytes] STAT |           0 |         0 |         0 |         0 |
| Memory write bandwidth [MBytes/s] STAT |           0 |         0 |         0 |         0 |
| Memory write data volume [GBytes] STAT |           0 |         0 |         0 |         0 |
|    Memory bandwidth [MBytes/s] STAT    |           0 |         0 |         0 |         0 |
|    Memory data volume [GBytes] STAT    |           0 |         0 |         0 |         0 |
+----------------------------------------+-------------+-----------+-----------+-----------+

(Note that I have removed core 1-23 from the first and the third tables since it'd make this post super crowded without adding much value. The trends for the rest of the cores is same as core 0).

I also printed the output log with "-V 3" and grepped for "not supported" following the previous posts, and I could not find any match there.

Am I missing something here?

TomTheBear commented 4 years ago

Which ACCESSMODE are you using? Does the folder /sys/devices/power/ exist? If yes, try ACCESSMODE=perf_event.

I don't see any usage errors, it's something with LIKWID internals or your system (not providing the required registers).

AMD is pretty silent when it comes to the data fabric counters in Zen and Zen2, so if your system does not provide the registers, I cannot do much from my side.

talnish commented 4 years ago

I was using accessdaemon. The folder that you mentioned does not exist. Nevertheless, I tried with perf_event ACCESSMODE, and still prints all 0s.

TomTheBear commented 4 years ago

Sure, if the folder does not exist, LIKWID cannot get the required information with ACCESSMODE=perf_event.

It might be some BIOS setting or the mainboard vendor disabled it.

For future cases, could you please post the output of likwid-perfctr -C 0,1 -g ENERGY -V 3 sleep 10

talnish commented 4 years ago 
CPU name:       AMD Ryzen Threadripper 3960X 24-Core Processor
CPU type:       AMD K17 (Zen2) architecture
CPU clock:      3.80 GHz
CPU family:     23
CPU model:      49
CPU short:      zen2
CPU stepping:   0
CPU features:   FP MMX SSE SSE2 HTT MMX RDTSCP MONITOR SSSE FMA SSE4.1 SSE4.2 AES AVX RDRAND SSE AVX2 RDSEED SSE3
CPU arch:       x86_64

DEBUG - [HPMinit:107] Adjusting functions for x86 architecture in direct mode
DEBUG - [access_x86_msr_init:206] Test for RDPMC for PMC counters returned 0
DEBUG - [access_x86_msr_init:211] Test for RDPMC for FIXED counters returned 0
DEBUG - [access_x86_msr_init:240] Opened MSR device /dev/cpu/0/msr for CPU 0
DEBUG - [HPMaddThread:143] Adding CPU 0 to access module
DEBUG - [access_x86_msr_read:291] Read MSR counter 0xC0010299 with RDMSR instruction on CPU 0
DEBUG - [access_x86_msr_read:291] Read MSR counter 0xC001029A with RDMSR instruction on CPU 0
DEBUG - [access_x86_msr_read:291] Read MSR counter 0xC001029B with RDMSR instruction on CPU 0
DEBUG - [access_x86_msr_init:240] Opened MSR device /dev/cpu/1/msr for CPU 1
DEBUG - [HPMaddThread:143] Adding CPU 1 to access module
Executing: sleep 10
DEBUG - [perfmon_addEventSet:1939] Currently 1 groups of 2 active
DEBUG - [perfgroup_readGroup:852] Reading group ENERGY from /usr/local/share/likwid/perfgroups/zen2/ENERGY.txt
DEBUG - [access_x86_msr_read:291] Read MSR counter 0xC00000E8 with RDMSR instruction on CPU 0
DEBUG - [perfmon_addEventSet:2120] Added event ACTUAL_CPU_CLOCK for counter FIXC1 to group 0
DEBUG - [access_x86_msr_read:291] Read MSR counter 0xC00000E7 with RDMSR instruction on CPU 0
DEBUG - [perfmon_addEventSet:2120] Added event MAX_CPU_CLOCK for counter FIXC2 to group 0
DEBUG - [access_x86_msr_read:291] Read MSR counter 0xC0010200 with RDMSR instruction on CPU 0
DEBUG - [checkAccess:278] Counter PMC0 has bits set (0x530076) but we are forced to overwrite them
DEBUG - [perfmon_addEventSet:2120] Added event RETIRED_INSTRUCTIONS for counter PMC0 to group 0
DEBUG - [access_x86_msr_read:291] Read MSR counter 0xC0010202 with RDMSR instruction on CPU 0
DEBUG - [checkAccess:278] Counter PMC1 has bits set (0x2000013ff45) but we are forced to overwrite them
DEBUG - [perfmon_addEventSet:2120] Added event CPU_CLOCKS_UNHALTED for counter PMC1 to group 0
DEBUG - [access_x86_msr_read:291] Read MSR counter 0xC001029A with RDMSR instruction on CPU 0
DEBUG - [perfmon_addEventSet:2120] Added event RAPL_CORE_ENERGY for counter PWR0 to group 0
DEBUG - [access_x86_msr_read:291] Read MSR counter 0xC001029B with RDMSR instruction on CPU 0
DEBUG - [perfmon_addEventSet:2120] Added event RAPL_PKG_ENERGY for counter PWR1 to group 0
DEBUG - [zen2_pmc_setup:104] SETUP_PMC [0] Register 0xC0010200 , Flags: 0x100C0
DEBUG - [access_x86_msr_write:308] Write MSR counter 0xC0010200 with WRMSR instruction on CPU 0 data 0x100C0
DEBUG - [zen2_pmc_setup:104] SETUP_PMC [0] Register 0xC0010202 , Flags: 0x10076
DEBUG - [access_x86_msr_write:308] Write MSR counter 0xC0010202 with WRMSR instruction on CPU 0 data 0x10076
DEBUG - [zen2_pmc_setup:104] SETUP_PMC [1] Register 0xC0010200 , Flags: 0x100C0
DEBUG - [access_x86_msr_write:308] Write MSR counter 0xC0010200 with WRMSR instruction on CPU 1 data 0x100C0
DEBUG - [zen2_pmc_setup:104] SETUP_PMC [1] Register 0xC0010202 , Flags: 0x10076
DEBUG - [access_x86_msr_write:308] Write MSR counter 0xC0010202 with WRMSR instruction on CPU 1 data 0x10076

DEBUG - [access_x86_msr_read:291] Read MSR counter 0xC00000E8 with RDMSR instruction on CPU 0
DEBUG - [perfmon_startCountersThread_zen2:296] START_FIXED [0] Register 0xC00000E8 , Flags: 0x32C7072F8E96A
DEBUG - [access_x86_msr_read:291] Read MSR counter 0xC00000E7 with RDMSR instruction on CPU 0
DEBUG - [perfmon_startCountersThread_zen2:296] START_FIXED [0] Register 0xC00000E7 , Flags: 0x573AF4E21EE10
DEBUG - [perfmon_startCountersThread_zen2:274] RESET_CTR [0] Register 0xC0010201 , Flags: 0x0
DEBUG - [access_x86_msr_write:308] Write MSR counter 0xC0010201 with WRMSR instruction on CPU 0 data 0x0
DEBUG - [access_x86_msr_read:291] Read MSR counter 0xC0010200 with RDMSR instruction on CPU 0
DEBUG - [perfmon_startCountersThread_zen2:277] READ_CTRL [0] Register 0xC0010200 , Flags: 0x100C0
DEBUG - [perfmon_startCountersThread_zen2:279] START_CTRL [0] Register 0xC0010200 , Flags: 0x4100C0
DEBUG - [access_x86_msr_write:308] Write MSR counter 0xC0010200 with WRMSR instruction on CPU 0 data 0x4100C0
DEBUG - [perfmon_startCountersThread_zen2:274] RESET_CTR [0] Register 0xC0010203 , Flags: 0x0
DEBUG - [access_x86_msr_write:308] Write MSR counter 0xC0010203 with WRMSR instruction on CPU 0 data 0x0
DEBUG - [access_x86_msr_read:291] Read MSR counter 0xC0010202 with RDMSR instruction on CPU 0
DEBUG - [perfmon_startCountersThread_zen2:277] READ_CTRL [0] Register 0xC0010202 , Flags: 0x10076
DEBUG - [perfmon_startCountersThread_zen2:279] START_CTRL [0] Register 0xC0010202 , Flags: 0x410076
DEBUG - [access_x86_msr_write:308] Write MSR counter 0xC0010202 with WRMSR instruction on CPU 0 data 0x410076
DEBUG - [access_x86_msr_read:291] Read MSR counter 0xC001029A with RDMSR instruction on CPU 0
DEBUG - [perfmon_startCountersThread_zen2:290] START_POWER [0] Register 0xC001029A , Flags: 0xB9B22722
DEBUG - [access_x86_msr_read:291] Read MSR counter 0xC001029B with RDMSR instruction on CPU 0
DEBUG - [perfmon_startCountersThread_zen2:290] START_POWER [0] Register 0xC001029B , Flags: 0x19D0F5D2
DEBUG - [access_x86_msr_read:291] Read MSR counter 0xC00000E8 with RDMSR instruction on CPU 1
DEBUG - [perfmon_startCountersThread_zen2:296] START_FIXED [1] Register 0xC00000E8 , Flags: 0x32CD00B93D88E
DEBUG - [access_x86_msr_read:291] Read MSR counter 0xC00000E7 with RDMSR instruction on CPU 1
DEBUG - [perfmon_startCountersThread_zen2:296] START_FIXED [1] Register 0xC00000E7 , Flags: 0x576415A139E50
DEBUG - [perfmon_startCountersThread_zen2:274] RESET_CTR [1] Register 0xC0010201 , Flags: 0x0
DEBUG - [access_x86_msr_write:308] Write MSR counter 0xC0010201 with WRMSR instruction on CPU 1 data 0x0
DEBUG - [access_x86_msr_read:291] Read MSR counter 0xC0010200 with RDMSR instruction on CPU 1
DEBUG - [perfmon_startCountersThread_zen2:277] READ_CTRL [1] Register 0xC0010200 , Flags: 0x100C0
DEBUG - [perfmon_startCountersThread_zen2:279] START_CTRL [1] Register 0xC0010200 , Flags: 0x4100C0
DEBUG - [access_x86_msr_write:308] Write MSR counter 0xC0010200 with WRMSR instruction on CPU 1 data 0x4100C0
DEBUG - [perfmon_startCountersThread_zen2:274] RESET_CTR [1] Register 0xC0010203 , Flags: 0x0
DEBUG - [access_x86_msr_write:308] Write MSR counter 0xC0010203 with WRMSR instruction on CPU 1 data 0x0
DEBUG - [access_x86_msr_read:291] Read MSR counter 0xC0010202 with RDMSR instruction on CPU 1
DEBUG - [perfmon_startCountersThread_zen2:277] READ_CTRL [1] Register 0xC0010202 , Flags: 0x10076
DEBUG - [perfmon_startCountersThread_zen2:279] START_CTRL [1] Register 0xC0010202 , Flags: 0x410076
DEBUG - [access_x86_msr_write:308] Write MSR counter 0xC0010202 with WRMSR instruction on CPU 1 data 0x410076
DEBUG - [access_x86_msr_read:291] Read MSR counter 0xC001029A with RDMSR instruction on CPU 1
DEBUG - [perfmon_startCountersThread_zen2:290] START_POWER [1] Register 0xC001029A , Flags: 0xB4E02D16
Sleeping longer as likwid_sleep() called without prior initialization
Sleeping longer as likwid_sleep() called without prior initialization
DEBUG - [access_x86_msr_read:291] Read MSR counter 0xC00000E8 with RDMSR instruction on CPU 0
DEBUG - [perfmon_stopCountersThread_zen2:385] STOP_FIXED [0] Register 0xC00000E8 , Flags: 0x32C755392B932
DEBUG - [access_x86_msr_read:291] Read MSR counter 0xC00000E7 with RDMSR instruction on CPU 0
DEBUG - [perfmon_stopCountersThread_zen2:385] STOP_FIXED [0] Register 0xC00000E7 , Flags: 0x573B7BAD17DCC
DEBUG - [access_x86_msr_read:291] Read MSR counter 0xC0010200 with RDMSR instruction on CPU 0
DEBUG - [perfmon_stopCountersThread_zen2:350] STOP_CTRL [0] Register 0xC0010200 , Flags: 0x100C0
DEBUG - [access_x86_msr_write:308] Write MSR counter 0xC0010200 with WRMSR instruction on CPU 0 data 0x100C0
DEBUG - [access_x86_msr_read:291] Read MSR counter 0xC0010201 with RDMSR instruction on CPU 0
DEBUG - [perfmon_stopCountersThread_zen2:353] READ_CTR [0] Register 0xC0010200 , Flags: 0xE63EC
DEBUG - [access_x86_msr_read:291] Read MSR counter 0xC0010202 with RDMSR instruction on CPU 0
DEBUG - [perfmon_stopCountersThread_zen2:350] STOP_CTRL [0] Register 0xC0010202 , Flags: 0x10076
DEBUG - [access_x86_msr_write:308] Write MSR counter 0xC0010202 with WRMSR instruction on CPU 0 data 0x10076
DEBUG - [access_x86_msr_read:291] Read MSR counter 0xC0010203 with RDMSR instruction on CPU 0
DEBUG - [perfmon_stopCountersThread_zen2:353] READ_CTR [0] Register 0xC0010202 , Flags: 0x6DFC2
DEBUG - [access_x86_msr_read:291] Read MSR counter 0xC001029A with RDMSR instruction on CPU 0
DEBUG - [perfmon_stopCountersThread_zen2:374] STOP_POWER [0] Register 0xC001029A , Flags: 0xB9B487DC
DEBUG - [access_x86_msr_read:291] Read MSR counter 0xC001029B with RDMSR instruction on CPU 0
DEBUG - [perfmon_stopCountersThread_zen2:374] STOP_POWER [0] Register 0xC001029B , Flags: 0x1BE1216E
DEBUG - [access_x86_msr_read:291] Read MSR counter 0xC00000E8 with RDMSR instruction on CPU 1
DEBUG - [perfmon_stopCountersThread_zen2:385] STOP_FIXED [1] Register 0xC00000E8 , Flags: 0x32CD4F4B81E27
DEBUG - [access_x86_msr_read:291] Read MSR counter 0xC00000E7 with RDMSR instruction on CPU 1
DEBUG - [perfmon_stopCountersThread_zen2:385] STOP_FIXED [1] Register 0xC00000E7 , Flags: 0x57649D57E71C8
DEBUG - [access_x86_msr_read:291] Read MSR counter 0xC0010200 with RDMSR instruction on CPU 1
DEBUG - [perfmon_stopCountersThread_zen2:350] STOP_CTRL [1] Register 0xC0010200 , Flags: 0x100C0
DEBUG - [access_x86_msr_write:308] Write MSR counter 0xC0010200 with WRMSR instruction on CPU 1 data 0x100C0
DEBUG - [access_x86_msr_read:291] Read MSR counter 0xC0010201 with RDMSR instruction on CPU 1
DEBUG - [perfmon_stopCountersThread_zen2:353] READ_CTR [1] Register 0xC0010200 , Flags: 0x4FEC9
DEBUG - [access_x86_msr_read:291] Read MSR counter 0xC0010202 with RDMSR instruction on CPU 1
DEBUG - [perfmon_stopCountersThread_zen2:350] STOP_CTRL [1] Register 0xC0010202 , Flags: 0x10076
DEBUG - [access_x86_msr_write:308] Write MSR counter 0xC0010202 with WRMSR instruction on CPU 1 data 0x10076
DEBUG - [access_x86_msr_read:291] Read MSR counter 0xC0010203 with RDMSR instruction on CPU 1
DEBUG - [perfmon_stopCountersThread_zen2:353] READ_CTR [1] Register 0xC0010202 , Flags: 0x79CD6
DEBUG - [access_x86_msr_read:291] Read MSR counter 0xC001029A with RDMSR instruction on CPU 1
DEBUG - [perfmon_stopCountersThread_zen2:374] STOP_POWER [1] Register 0xC001029A , Flags: 0xB4E29012

Group 1: ENERGY
+----------------------+---------+-------------+-------------+
|         Event        | Counter |    Core 0   |    Core 1   |
+----------------------+---------+-------------+-------------+
|   ACTUAL_CPU_CLOCK   |  FIXC1  | 20948045768 | 21091337625 |
|     MAX_CPU_CLOCK    |  FIXC2  | 36183183292 | 36430336888 |
| RETIRED_INSTRUCTIONS |   PMC0  |      943084 |      327369 |
|  CPU_CLOCKS_UNHALTED |   PMC1  |      450498 |      498902 |
|   RAPL_CORE_ENERGY   |   PWR0  |      2.3778 |      2.3867 |
|    RAPL_PKG_ENERGY   |   PWR1  |    528.1703 |           0 |
+----------------------+---------+-------------+-------------+

+---------------------------+---------+-------------+-------------+-------------+--------------+
|           Event           | Counter |     Sum     |     Min     |     Max     |      Avg     |
+---------------------------+---------+-------------+-------------+-------------+--------------+
|   ACTUAL_CPU_CLOCK STAT   |  FIXC1  | 42039383393 | 20948045768 | 21091337625 | 2.101969e+10 |
|     MAX_CPU_CLOCK STAT    |  FIXC2  | 72613520180 | 36183183292 | 36430336888 |  36306760090 |
| RETIRED_INSTRUCTIONS STAT |   PMC0  |     1270453 |      327369 |      943084 |  635226.5000 |
|  CPU_CLOCKS_UNHALTED STAT |   PMC1  |      949400 |      450498 |      498902 |       474700 |
|   RAPL_CORE_ENERGY STAT   |   PWR0  |      4.7645 |      2.3778 |      2.3867 |       2.3822 |
|    RAPL_PKG_ENERGY STAT   |   PWR1  |    528.1703 |           0 |    528.1703 |     264.0851 |
+---------------------------+---------+-------------+-------------+-------------+--------------+

+----------------------+-----------+-----------+
|        Metric        |   Core 0  |   Core 1  |
+----------------------+-----------+-----------+
|  Runtime (RDTSC) [s] |   10.0027 |   10.0027 |
| Runtime unhalted [s] |    5.5132 |    5.5509 |
|      Clock [MHz]     | 2199.7852 | 2199.8064 |
|          CPI         |    0.4777 |    1.5240 |
|    Energy Core [J]   |    2.3778 |    2.3867 |
|    Power Core [W]    |    0.2377 |    0.2386 |
|    Energy PKG [J]    |  528.1703 |         0 |
|     Power PKG [W]    |   52.8029 |         0 |
+----------------------+-----------+-----------+

+---------------------------+-----------+-----------+-----------+-----------+
|           Metric          |    Sum    |    Min    |    Max    |    Avg    |
+---------------------------+-----------+-----------+-----------+-----------+
|  Runtime (RDTSC) [s] STAT |   20.0054 |   10.0027 |   10.0027 |   10.0027 |
| Runtime unhalted [s] STAT |   11.0641 |    5.5132 |    5.5509 |    5.5320 |
|      Clock [MHz] STAT     | 4399.5916 | 2199.7852 | 2199.8064 | 2199.7958 |
|          CPI STAT         |    2.0017 |    0.4777 |    1.5240 |    1.0009 |
|    Energy Core [J] STAT   |    4.7645 |    2.3778 |    2.3867 |    2.3822 |
|    Power Core [W] STAT    |    0.4763 |    0.2377 |    0.2386 |    0.2382 |
|    Energy PKG [J] STAT    |  528.1703 |         0 |  528.1703 |  264.0851 |
|     Power PKG [W] STAT    |   52.8029 |         0 |   52.8029 |   26.4015 |
+---------------------------+-----------+-----------+-----------+-----------+

DEBUG - [access_x86_msr_read:291] Read MSR counter 0xC0010015 with RDMSR instruction on CPU 0
DEBUG - [access_x86_msr_write:308] Write MSR counter 0xC0010015 with WRMSR instruction on CPU 0 data 0x10B000011
DEBUG - [access_x86_msr_read:291] Read MSR counter 0xC0010015 with RDMSR instruction on CPU 0
DEBUG - [access_x86_msr_write:308] Write MSR counter 0xC0010015 with WRMSR instruction on CPU 0 data 0x10B000011
DEBUG - [perfmon_finalizeCountersThread_zen2:512] CLEAR_CTRL [0] Register 0xC0010200 , Flags: 0x0
DEBUG - [access_x86_msr_write:308] Write MSR counter 0xC0010200 with WRMSR instruction on CPU 0 data 0x0
DEBUG - [perfmon_finalizeCountersThread_zen2:517] CLEAR_CTR [0] Register 0xC0010201 , Flags: 0x0
DEBUG - [access_x86_msr_write:308] Write MSR counter 0xC0010201 with WRMSR instruction on CPU 0 data 0x0
DEBUG - [perfmon_finalizeCountersThread_zen2:512] CLEAR_CTRL [0] Register 0xC0010202 , Flags: 0x0
DEBUG - [access_x86_msr_write:308] Write MSR counter 0xC0010202 with WRMSR instruction on CPU 0 data 0x0
DEBUG - [perfmon_finalizeCountersThread_zen2:517] CLEAR_CTR [0] Register 0xC0010203 , Flags: 0x0
DEBUG - [access_x86_msr_write:308] Write MSR counter 0xC0010203 with WRMSR instruction on CPU 0 data 0x0
DEBUG - [access_x86_msr_read:291] Read MSR counter 0xC0010015 with RDMSR instruction on CPU 1
DEBUG - [access_x86_msr_write:308] Write MSR counter 0xC0010015 with WRMSR instruction on CPU 1 data 0x10B000011
DEBUG - [access_x86_msr_read:291] Read MSR counter 0xC0010015 with RDMSR instruction on CPU 1
DEBUG - [access_x86_msr_write:308] Write MSR counter 0xC0010015 with WRMSR instruction on CPU 1 data 0x10B000011
DEBUG - [perfmon_finalizeCountersThread_zen2:512] CLEAR_CTRL [1] Register 0xC0010200 , Flags: 0x0
DEBUG - [access_x86_msr_write:308] Write MSR counter 0xC0010200 with WRMSR instruction on CPU 1 data 0x0
DEBUG - [perfmon_finalizeCountersThread_zen2:517] CLEAR_CTR [1] Register 0xC0010201 , Flags: 0x0
DEBUG - [access_x86_msr_write:308] Write MSR counter 0xC0010201 with WRMSR instruction on CPU 1 data 0x0
DEBUG - [perfmon_finalizeCountersThread_zen2:512] CLEAR_CTRL [1] Register 0xC0010202 , Flags: 0x0
DEBUG - [access_x86_msr_write:308] Write MSR counter 0xC0010202 with WRMSR instruction on CPU 1 data 0x0
DEBUG - [perfmon_finalizeCountersThread_zen2:517] CLEAR_CTR [1] Register 0xC0010203 , Flags: 0x0
DEBUG - [access_x86_msr_write:308] Write MSR counter 0xC0010203 with WRMSR instruction on CPU 1 data 0x0
DEBUG - [HPMfinalize:170] Removing CPU 0 from access module
DEBUG - [HPMfinalize:170] Removing CPU 1 from access module
TomTheBear commented 4 years ago

Here you have values unequal to zero for the energy events although it's just a sleep.

You are using the ACCESSMODE=direct which should generally work but is less tested as accessdaemon or perf_event.

talnish commented 4 years ago

I have tried most other metrics and they show non-zero values (i.e., TLB, CACHE, BRANCH, ENERGY); it's only the MEM that's showing bandwidth numbers as 0.

TomTheBear commented 4 years ago

I requested the wrong output, sorry. Interesting would be the MEM group, since you have problems with it: likwid-perfctr -C 0,1 -g MEM -V 3 <exec> with an <exec> that does some memory operations like STREAM or likwid-bench.

talnish commented 4 years ago 
CPU name:   AMD Ryzen Threadripper 3960X 24-Core Processor 
CPU type:   AMD K17 (Zen2) architecture
CPU clock:  3.80 GHz
CPU family: 23
CPU model:  49
CPU short:  zen2
CPU stepping:   0
CPU features:   FP MMX SSE SSE2 HTT MMX RDTSCP MONITOR SSSE FMA SSE4.1 SSE4.2 AES AVX RDRAND SSE AVX2 RDSEED SSE3 
CPU arch:   x86_64

DEBUG - [HPMinit:107] Adjusting functions for x86 architecture in direct mode
DEBUG - [access_x86_msr_init:206] Test for RDPMC for PMC counters returned 0
DEBUG - [access_x86_msr_init:211] Test for RDPMC for FIXED counters returned 0
DEBUG - [access_x86_msr_init:240] Opened MSR device /dev/cpu/0/msr for CPU 0
DEBUG - [HPMaddThread:143] Adding CPU 0 to access module
DEBUG - [access_x86_msr_read:291] Read MSR counter 0xC0010299 with RDMSR instruction on CPU 0
DEBUG - [access_x86_msr_read:291] Read MSR counter 0xC001029A with RDMSR instruction on CPU 0
DEBUG - [access_x86_msr_read:291] Read MSR counter 0xC001029B with RDMSR instruction on CPU 0
DEBUG - [access_x86_msr_init:240] Opened MSR device /dev/cpu/1/msr for CPU 1
DEBUG - [HPMaddThread:143] Adding CPU 1 to access module

DEBUG - [perfmon_addEventSet:1939] Currently 1 groups of 2 active
DEBUG - [perfgroup_readGroup:852] Reading group MEM from /usr/local/share/likwid/perfgroups/zen2/MEM.txt
DEBUG - [access_x86_msr_read:291] Read MSR counter 0xC00000E8 with RDMSR instruction on CPU 0
DEBUG - [perfmon_addEventSet:2120] Added event ACTUAL_CPU_CLOCK for counter FIXC1 to group 0
DEBUG - [access_x86_msr_read:291] Read MSR counter 0xC00000E7 with RDMSR instruction on CPU 0
DEBUG - [perfmon_addEventSet:2120] Added event MAX_CPU_CLOCK for counter FIXC2 to group 0
DEBUG - [access_x86_msr_read:291] Read MSR counter 0xC0010200 with RDMSR instruction on CPU 0
DEBUG - [access_x86_msr_write:308] Write MSR counter 0xC0010200 with WRMSR instruction on CPU 0 data 0x0
DEBUG - [perfmon_addEventSet:2120] Added event RETIRED_INSTRUCTIONS for counter PMC0 to group 0
DEBUG - [access_x86_msr_read:291] Read MSR counter 0xC0010202 with RDMSR instruction on CPU 0
DEBUG - [access_x86_msr_write:308] Write MSR counter 0xC0010202 with WRMSR instruction on CPU 0 data 0x0
DEBUG - [perfmon_addEventSet:2120] Added event CPU_CLOCKS_UNHALTED for counter PMC1 to group 0
DEBUG - [access_x86_msr_read:291] Read MSR counter 0xC0010240 with RDMSR instruction on CPU 0
DEBUG - [access_x86_msr_write:308] Write MSR counter 0xC0010240 with WRMSR instruction on CPU 0 data 0x0
DEBUG - [perfmon_addEventSet:2120] Added event DATA_FROM_LOCAL_DRAM_CHANNEL for counter DFC0 to group 0
DEBUG - [access_x86_msr_read:291] Read MSR counter 0xC0010242 with RDMSR instruction on CPU 0
DEBUG - [access_x86_msr_write:308] Write MSR counter 0xC0010242 with WRMSR instruction on CPU 0 data 0x0
DEBUG - [perfmon_addEventSet:2120] Added event DATA_TO_LOCAL_DRAM_CHANNEL for counter DFC1 to group 0
DEBUG - [zen2_pmc_setup:104] SETUP_PMC [0] Register 0xC0010200 , Flags: 0x100C0 
DEBUG - [access_x86_msr_write:308] Write MSR counter 0xC0010200 with WRMSR instruction on CPU 0 data 0x100C0
DEBUG - [zen2_pmc_setup:104] SETUP_PMC [0] Register 0xC0010202 , Flags: 0x10076 
DEBUG - [access_x86_msr_write:308] Write MSR counter 0xC0010202 with WRMSR instruction on CPU 0 data 0x10076
DEBUG - [zen2_uncore_setup:174] SETUP_DF [0] Register 0xC0010240 , Flags: 0x3807 
DEBUG - [access_x86_msr_write:308] Write MSR counter 0xC0010240 with WRMSR instruction on CPU 0 data 0x3807
DEBUG - [zen2_uncore_setup:174] SETUP_DF [0] Register 0xC0010242 , Flags: 0x3847 
DEBUG - [access_x86_msr_write:308] Write MSR counter 0xC0010242 with WRMSR instruction on CPU 0 data 0x3847
DEBUG - [zen2_pmc_setup:104] SETUP_PMC [1] Register 0xC0010200 , Flags: 0x100C0 
DEBUG - [access_x86_msr_write:308] Write MSR counter 0xC0010200 with WRMSR instruction on CPU 1 data 0x100C0
DEBUG - [zen2_pmc_setup:104] SETUP_PMC [1] Register 0xC0010202 , Flags: 0x10076 
DEBUG - [access_x86_msr_write:308] Write MSR counter 0xC0010202 with WRMSR instruction on CPU 1 data 0x10076

DEBUG - [access_x86_msr_read:291] Read MSR counter 0xC00000E8 with RDMSR instruction on CPU 0
DEBUG - [perfmon_startCountersThread_zen2:296] START_FIXED [0] Register 0xC00000E8 , Flags: 0x7A7F97963D2BE 
DEBUG - [access_x86_msr_read:291] Read MSR counter 0xC00000E7 with RDMSR instruction on CPU 0
DEBUG - [perfmon_startCountersThread_zen2:296] START_FIXED [0] Register 0xC00000E7 , Flags: 0xD2CB4DE281A38 
DEBUG - [perfmon_startCountersThread_zen2:274] RESET_CTR [0] Register 0xC0010201 , Flags: 0x0 
DEBUG - [access_x86_msr_write:308] Write MSR counter 0xC0010201 with WRMSR instruction on CPU 0 data 0x0
DEBUG - [access_x86_msr_read:291] Read MSR counter 0xC0010200 with RDMSR instruction on CPU 0
DEBUG - [perfmon_startCountersThread_zen2:277] READ_CTRL [0] Register 0xC0010200 , Flags: 0x100C0 
DEBUG - [perfmon_startCountersThread_zen2:279] START_CTRL [0] Register 0xC0010200 , Flags: 0x4100C0 
DEBUG - [access_x86_msr_write:308] Write MSR counter 0xC0010200 with WRMSR instruction on CPU 0 data 0x4100C0
DEBUG - [perfmon_startCountersThread_zen2:274] RESET_CTR [0] Register 0xC0010203 , Flags: 0x0 
DEBUG - [access_x86_msr_write:308] Write MSR counter 0xC0010203 with WRMSR instruction on CPU 0 data 0x0
DEBUG - [access_x86_msr_read:291] Read MSR counter 0xC0010202 with RDMSR instruction on CPU 0
DEBUG - [perfmon_startCountersThread_zen2:277] READ_CTRL [0] Register 0xC0010202 , Flags: 0x10076 
DEBUG - [perfmon_startCountersThread_zen2:279] START_CTRL [0] Register 0xC0010202 , Flags: 0x410076 
DEBUG - [access_x86_msr_write:308] Write MSR counter 0xC0010202 with WRMSR instruction on CPU 0 data 0x410076
DEBUG - [perfmon_startCountersThread_zen2:274] RESET_CTR [0] Register 0xC0010241 , Flags: 0x0 
DEBUG - [access_x86_msr_write:308] Write MSR counter 0xC0010241 with WRMSR instruction on CPU 0 data 0x0
DEBUG - [access_x86_msr_read:291] Read MSR counter 0xC0010240 with RDMSR instruction on CPU 0
DEBUG - [perfmon_startCountersThread_zen2:277] READ_CTRL [0] Register 0xC0010240 , Flags: 0x3807 
DEBUG - [perfmon_startCountersThread_zen2:279] START_CTRL [0] Register 0xC0010240 , Flags: 0x403807 
DEBUG - [access_x86_msr_write:308] Write MSR counter 0xC0010240 with WRMSR instruction on CPU 0 data 0x403807
DEBUG - [perfmon_startCountersThread_zen2:274] RESET_CTR [0] Register 0xC0010243 , Flags: 0x0 
DEBUG - [access_x86_msr_write:308] Write MSR counter 0xC0010243 with WRMSR instruction on CPU 0 data 0x0
DEBUG - [access_x86_msr_read:291] Read MSR counter 0xC0010242 with RDMSR instruction on CPU 0
DEBUG - [perfmon_startCountersThread_zen2:277] READ_CTRL [0] Register 0xC0010242 , Flags: 0x3847 
DEBUG - [perfmon_startCountersThread_zen2:279] START_CTRL [0] Register 0xC0010242 , Flags: 0x403847 
DEBUG - [access_x86_msr_write:308] Write MSR counter 0xC0010242 with WRMSR instruction on CPU 0 data 0x403847
DEBUG - [access_x86_msr_read:291] Read MSR counter 0xC00000E8 with RDMSR instruction on CPU 1
DEBUG - [perfmon_startCountersThread_zen2:296] START_FIXED [1] Register 0xC00000E8 , Flags: 0x7BB2E62203A32 
DEBUG - [access_x86_msr_read:291] Read MSR counter 0xC00000E7 with RDMSR instruction on CPU 1
DEBUG - [perfmon_startCountersThread_zen2:296] START_FIXED [1] Register 0xC00000E7 , Flags: 0xD4FFE284075F2 
DEBUG - [perfmon_startCountersThread_zen2:274] RESET_CTR [1] Register 0xC0010201 , Flags: 0x0 
DEBUG - [access_x86_msr_write:308] Write MSR counter 0xC0010201 with WRMSR instruction on CPU 1 data 0x0
DEBUG - [access_x86_msr_read:291] Read MSR counter 0xC0010200 with RDMSR instruction on CPU 1
DEBUG - [perfmon_startCountersThread_zen2:277] READ_CTRL [1] Register 0xC0010200 , Flags: 0x100C0 
DEBUG - [perfmon_startCountersThread_zen2:279] START_CTRL [1] Register 0xC0010200 , Flags: 0x4100C0 
DEBUG - [access_x86_msr_write:308] Write MSR counter 0xC0010200 with WRMSR instruction on CPU 1 data 0x4100C0
DEBUG - [perfmon_startCountersThread_zen2:274] RESET_CTR [1] Register 0xC0010203 , Flags: 0x0 
DEBUG - [access_x86_msr_write:308] Write MSR counter 0xC0010203 with WRMSR instruction on CPU 1 data 0x0
DEBUG - [access_x86_msr_read:291] Read MSR counter 0xC0010202 with RDMSR instruction on CPU 1
DEBUG - [perfmon_startCountersThread_zen2:277] READ_CTRL [1] Register 0xC0010202 , Flags: 0x10076 
DEBUG - [perfmon_startCountersThread_zen2:279] START_CTRL [1] Register 0xC0010202 , Flags: 0x410076 
DEBUG - [access_x86_msr_write:308] Write MSR counter 0xC0010202 with WRMSR instruction on CPU 1 data 0x410076

DEBUG - [access_x86_msr_read:291] Read MSR counter 0xC00000E8 with RDMSR instruction on CPU 0
DEBUG - [perfmon_stopCountersThread_zen2:385] STOP_FIXED [0] Register 0xC00000E8 , Flags: 0x7A7F9796581EE 
DEBUG - [access_x86_msr_read:291] Read MSR counter 0xC00000E7 with RDMSR instruction on CPU 0
DEBUG - [perfmon_stopCountersThread_zen2:385] STOP_FIXED [0] Register 0xC00000E7 , Flags: 0xD2CB4DE2B0B8A 
DEBUG - [access_x86_msr_read:291] Read MSR counter 0xC0010200 with RDMSR instruction on CPU 0
DEBUG - [perfmon_stopCountersThread_zen2:350] STOP_CTRL [0] Register 0xC0010200 , Flags: 0x100C0 
DEBUG - [access_x86_msr_write:308] Write MSR counter 0xC0010200 with WRMSR instruction on CPU 0 data 0x100C0
DEBUG - [access_x86_msr_read:291] Read MSR counter 0xC0010201 with RDMSR instruction on CPU 0
DEBUG - [perfmon_stopCountersThread_zen2:353] READ_CTR [0] Register 0xC0010200 , Flags: 0x0 
DEBUG - [access_x86_msr_read:291] Read MSR counter 0xC0010202 with RDMSR instruction on CPU 0
DEBUG - [perfmon_stopCountersThread_zen2:350] STOP_CTRL [0] Register 0xC0010202 , Flags: 0x10076 
DEBUG - [access_x86_msr_write:308] Write MSR counter 0xC0010202 with WRMSR instruction on CPU 0 data 0x10076
DEBUG - [access_x86_msr_read:291] Read MSR counter 0xC0010203 with RDMSR instruction on CPU 0
DEBUG - [perfmon_stopCountersThread_zen2:353] READ_CTR [0] Register 0xC0010202 , Flags: 0x0 
DEBUG - [access_x86_msr_read:291] Read MSR counter 0xC0010240 with RDMSR instruction on CPU 0
DEBUG - [perfmon_stopCountersThread_zen2:350] STOP_CTRL [0] Register 0xC0010240 , Flags: 0x3807 
DEBUG - [access_x86_msr_write:308] Write MSR counter 0xC0010240 with WRMSR instruction on CPU 0 data 0x3807
DEBUG - [access_x86_msr_read:291] Read MSR counter 0xC0010241 with RDMSR instruction on CPU 0
DEBUG - [perfmon_stopCountersThread_zen2:353] READ_CTR [0] Register 0xC0010240 , Flags: 0x0 
DEBUG - [access_x86_msr_read:291] Read MSR counter 0xC0010242 with RDMSR instruction on CPU 0
DEBUG - [perfmon_stopCountersThread_zen2:350] STOP_CTRL [0] Register 0xC0010242 , Flags: 0x3847 
DEBUG - [access_x86_msr_write:308] Write MSR counter 0xC0010242 with WRMSR instruction on CPU 0 data 0x3847
DEBUG - [access_x86_msr_read:291] Read MSR counter 0xC0010243 with RDMSR instruction on CPU 0
DEBUG - [perfmon_stopCountersThread_zen2:353] READ_CTR [0] Register 0xC0010242 , Flags: 0x0 
DEBUG - [access_x86_msr_read:291] Read MSR counter 0xC00000E8 with RDMSR instruction on CPU 1
DEBUG - [perfmon_stopCountersThread_zen2:385] STOP_FIXED [1] Register 0xC00000E8 , Flags: 0x7BB2E6249E4BF 
DEBUG - [access_x86_msr_read:291] Read MSR counter 0xC00000E7 with RDMSR instruction on CPU 1
DEBUG - [perfmon_stopCountersThread_zen2:385] STOP_FIXED [1] Register 0xC00000E7 , Flags: 0xD4FFE28887016 
DEBUG - [access_x86_msr_read:291] Read MSR counter 0xC0010200 with RDMSR instruction on CPU 1
DEBUG - [perfmon_stopCountersThread_zen2:350] STOP_CTRL [1] Register 0xC0010200 , Flags: 0x100C0 
DEBUG - [access_x86_msr_write:308] Write MSR counter 0xC0010200 with WRMSR instruction on CPU 1 data 0x100C0
DEBUG - [access_x86_msr_read:291] Read MSR counter 0xC0010201 with RDMSR instruction on CPU 1
DEBUG - [perfmon_stopCountersThread_zen2:353] READ_CTR [1] Register 0xC0010200 , Flags: 0x44BAB 
DEBUG - [access_x86_msr_read:291] Read MSR counter 0xC0010202 with RDMSR instruction on CPU 1
DEBUG - [perfmon_stopCountersThread_zen2:350] STOP_CTRL [1] Register 0xC0010202 , Flags: 0x10076 
DEBUG - [access_x86_msr_write:308] Write MSR counter 0xC0010202 with WRMSR instruction on CPU 1 data 0x10076
DEBUG - [access_x86_msr_read:291] Read MSR counter 0xC0010203 with RDMSR instruction on CPU 1
DEBUG - [perfmon_stopCountersThread_zen2:353] READ_CTR [1] Register 0xC0010202 , Flags: 0x79EBB 

Group 1: MEM
+------------------------------+---------+--------+---------+
|             Event            | Counter | Core 0 |  Core 1 |
+------------------------------+---------+--------+---------+
|       ACTUAL_CPU_CLOCK       |  FIXC1  | 110384 | 2730637 |
|         MAX_CPU_CLOCK        |  FIXC2  | 192850 | 4717092 |
|     RETIRED_INSTRUCTIONS     |   PMC0  |      0 |  281515 |
|      CPU_CLOCKS_UNHALTED     |   PMC1  |      0 |  499387 |
| DATA_FROM_LOCAL_DRAM_CHANNEL |   DFC0  |      0 |       0 |
|  DATA_TO_LOCAL_DRAM_CHANNEL  |   DFC1  |      0 |       0 |
+------------------------------+---------+--------+---------+

+-----------------------------------+---------+---------+--------+---------+--------------+
|               Event               | Counter |   Sum   |   Min  |   Max   |      Avg     |
+-----------------------------------+---------+---------+--------+---------+--------------+
|       ACTUAL_CPU_CLOCK STAT       |  FIXC1  | 2841021 | 110384 | 2730637 | 1.420510e+06 |
|         MAX_CPU_CLOCK STAT        |  FIXC2  | 4909942 | 192850 | 4717092 |      2454971 |
|     RETIRED_INSTRUCTIONS STAT     |   PMC0  |  281515 |      0 |  281515 |  140757.5000 |
|      CPU_CLOCKS_UNHALTED STAT     |   PMC1  |  499387 |      0 |  499387 |  249693.5000 |
| DATA_FROM_LOCAL_DRAM_CHANNEL STAT |   DFC0  |       0 |      0 |       0 |            0 |
|  DATA_TO_LOCAL_DRAM_CHANNEL STAT  |   DFC1  |       0 |      0 |       0 |            0 |
+-----------------------------------+---------+---------+--------+---------+--------------+

+-----------------------------------+--------------+-----------+
|               Metric              |    Core 0    |   Core 1  |
+-----------------------------------+--------------+-----------+
|        Runtime (RDTSC) [s]        |       0.0017 |    0.0017 |
|        Runtime unhalted [s]       | 2.904888e-05 |    0.0007 |
|            Clock [MHz]            |    2175.0196 | 2199.7144 |
|                CPI                |       -      |    1.7739 |
|  Memory read bandwidth [MBytes/s] |            0 |         0 |
|  Memory read data volume [GBytes] |            0 |         0 |
| Memory write bandwidth [MBytes/s] |            0 |         0 |
| Memory write data volume [GBytes] |            0 |         0 |
|    Memory bandwidth [MBytes/s]    |            0 |         0 |
|    Memory data volume [GBytes]    |            0 |         0 |
+-----------------------------------+--------------+-----------+

+----------------------------------------+-----------+--------------+-----------+-----------+
|                 Metric                 |    Sum    |      Min     |    Max    |    Avg    |
+----------------------------------------+-----------+--------------+-----------+-----------+
|        Runtime (RDTSC) [s] STAT        |    0.0034 |       0.0017 |    0.0017 |    0.0017 |
|        Runtime unhalted [s] STAT       |    0.0007 | 2.904888e-05 |    0.0007 |    0.0004 |
|            Clock [MHz] STAT            | 4374.7340 |    2175.0196 | 2199.7144 | 2187.3670 |
|                CPI STAT                |    1.7739 |       1.7739 |    1.7739 |    0.8870 |
|  Memory read bandwidth [MBytes/s] STAT |         0 |            0 |         0 |         0 |
|  Memory read data volume [GBytes] STAT |         0 |            0 |         0 |         0 |
| Memory write bandwidth [MBytes/s] STAT |         0 |            0 |         0 |         0 |
| Memory write data volume [GBytes] STAT |         0 |            0 |         0 |         0 |
|    Memory bandwidth [MBytes/s] STAT    |         0 |            0 |         0 |         0 |
|    Memory data volume [GBytes] STAT    |         0 |            0 |         0 |         0 |
+----------------------------------------+-----------+--------------+-----------+-----------+

DEBUG - [access_x86_msr_read:291] Read MSR counter 0xC0010015 with RDMSR instruction on CPU 0
DEBUG - [access_x86_msr_write:308] Write MSR counter 0xC0010015 with WRMSR instruction on CPU 0 data 0x10B000011
DEBUG - [access_x86_msr_read:291] Read MSR counter 0xC0010015 with RDMSR instruction on CPU 0
DEBUG - [access_x86_msr_write:308] Write MSR counter 0xC0010015 with WRMSR instruction on CPU 0 data 0x10B000011
DEBUG - [perfmon_finalizeCountersThread_zen2:512] CLEAR_CTRL [0] Register 0xC0010200 , Flags: 0x0 
DEBUG - [access_x86_msr_write:308] Write MSR counter 0xC0010200 with WRMSR instruction on CPU 0 data 0x0
DEBUG - [perfmon_finalizeCountersThread_zen2:517] CLEAR_CTR [0] Register 0xC0010201 , Flags: 0x0 
DEBUG - [access_x86_msr_write:308] Write MSR counter 0xC0010201 with WRMSR instruction on CPU 0 data 0x0
DEBUG - [perfmon_finalizeCountersThread_zen2:512] CLEAR_CTRL [0] Register 0xC0010202 , Flags: 0x0 
DEBUG - [access_x86_msr_write:308] Write MSR counter 0xC0010202 with WRMSR instruction on CPU 0 data 0x0
DEBUG - [perfmon_finalizeCountersThread_zen2:517] CLEAR_CTR [0] Register 0xC0010203 , Flags: 0x0 
DEBUG - [access_x86_msr_write:308] Write MSR counter 0xC0010203 with WRMSR instruction on CPU 0 data 0x0
DEBUG - [perfmon_finalizeCountersThread_zen2:512] CLEAR_CTRL [0] Register 0xC0010240 , Flags: 0x0 
DEBUG - [access_x86_msr_write:308] Write MSR counter 0xC0010240 with WRMSR instruction on CPU 0 data 0x0
DEBUG - [perfmon_finalizeCountersThread_zen2:517] CLEAR_CTR [0] Register 0xC0010241 , Flags: 0x0 
DEBUG - [access_x86_msr_write:308] Write MSR counter 0xC0010241 with WRMSR instruction on CPU 0 data 0x0
DEBUG - [perfmon_finalizeCountersThread_zen2:512] CLEAR_CTRL [0] Register 0xC0010242 , Flags: 0x0 
DEBUG - [access_x86_msr_write:308] Write MSR counter 0xC0010242 with WRMSR instruction on CPU 0 data 0x0
DEBUG - [perfmon_finalizeCountersThread_zen2:517] CLEAR_CTR [0] Register 0xC0010243 , Flags: 0x0 
DEBUG - [access_x86_msr_write:308] Write MSR counter 0xC0010243 with WRMSR instruction on CPU 0 data 0x0
DEBUG - [access_x86_msr_read:291] Read MSR counter 0xC0010015 with RDMSR instruction on CPU 1
DEBUG - [access_x86_msr_write:308] Write MSR counter 0xC0010015 with WRMSR instruction on CPU 1 data 0x10B000011
DEBUG - [access_x86_msr_read:291] Read MSR counter 0xC0010015 with RDMSR instruction on CPU 1
DEBUG - [access_x86_msr_write:308] Write MSR counter 0xC0010015 with WRMSR instruction on CPU 1 data 0x10B000011
DEBUG - [perfmon_finalizeCountersThread_zen2:512] CLEAR_CTRL [1] Register 0xC0010200 , Flags: 0x0 
DEBUG - [access_x86_msr_write:308] Write MSR counter 0xC0010200 with WRMSR instruction on CPU 1 data 0x0
DEBUG - [perfmon_finalizeCountersThread_zen2:517] CLEAR_CTR [1] Register 0xC0010201 , Flags: 0x0 
DEBUG - [access_x86_msr_write:308] Write MSR counter 0xC0010201 with WRMSR instruction on CPU 1 data 0x0
DEBUG - [perfmon_finalizeCountersThread_zen2:512] CLEAR_CTRL [1] Register 0xC0010202 , Flags: 0x0 
DEBUG - [access_x86_msr_write:308] Write MSR counter 0xC0010202 with WRMSR instruction on CPU 1 data 0x0
DEBUG - [perfmon_finalizeCountersThread_zen2:517] CLEAR_CTR [1] Register 0xC0010203 , Flags: 0x0 
DEBUG - [access_x86_msr_write:308] Write MSR counter 0xC0010203 with WRMSR instruction on CPU 1 data 0x0
DEBUG - [HPMfinalize:170] Removing CPU 0 from access module
DEBUG - [HPMfinalize:170] Removing CPU 1 from access module
TomTheBear commented 4 years ago

Unfortunately, I cannot see any problems from my side. The configuration registers are properly set up, started and stopped. The counter register is zero'd at the beginning but after counting, the register is still zero.

Have you tried a different access mode in LIKWID? Is there a folder amd_df in /sys/devices?

talnish commented 4 years ago

Yes, I do have amd_df folder in /sys/devices.

One thing I noticed is that no matter what ACCESSMODE I set in config.mk, I always see the following line:

Adjusting functions for x86 architecture in direct mode

Am I missing something here? Do I need to change something else in addition to changing ACCESSMODE in config.mk to run with other modes?

TomTheBear commented 4 years ago

The ACCESSMODE thing shouldn't happen. There are only the three valid settings:

Everything else should be set automatically by the build system

talnish commented 4 years ago

I noticed that once I set ACCESSMODE to something, it doesn't change if I change it from the config file and do make clean; make; sudo make install. I tried it using a different machine, set ACCESSMODE to accessdaemon, and the it stays at that even if I change it to direct and do make clean; make; sudo make install. Is this an expected behavior or am I missing a command here?

FYI, even with accessdaemon, the bandwidth numbers are still 0.

TomTheBear commented 4 years ago

Please do make distclean after changing settings in config.mk

Does it work with ACCESSMODE=perf_event?

talnish commented 4 years ago

Thanks for the pointer.

I double checked all the modes again - the bandwidth calculation shows non-zero numbers for perf_event and zero numbers for accessdaemon and direct.

TomTheBear commented 4 years ago

I'll check that. Thanks for testing all three backends.

TomTheBear commented 4 years ago

I cannot reproduce it. I get non-zero values for direct, accessdaemon and perf_event on our test system but I'll try on another Zen2 node as soon as I have access.