Help on family 14H - Githubissues

wmarkow commented 5 years ago

Hi @kevinlekiller,

I have a HP T610 Plus thin client device which has a AMD G-T56N processor on board. It is a 14H family and it is not supported by your amdctl software. However I was able to make some changes in amdctl.c file, so at least the amdctl -g will show me some generic output of what I have inside of my processor. At first I took a look at the https://www.amd.com/system/files/TechDocs/43170_14h_Mod_00h-0Fh_BKDG.pdf documentation and according to this, I have added some bit definitions in your checkFamily() method:

    case AMD14H:
        printf("AMD14H not fully supported. Test mode will be automatically enabled!");
        testMode = 1;
        CPU_VID_BITS = "15:9";
        CPU_DID_BITS = "8:0";
        // there is no FID on AMD14H, let's read it from DID for now
        CPU_FID_BITS = "8:0";
        break;

It looks like there is no FID bits in 14H (or it was moved or replaced by something else), so I faked them to be the same as DID bits. Moreover I have noticed that the data stored in a DID bits are in a different format (as for example in 15H family): they contain a decimal and fraction parts, so the calculations of the divider are a bit complex. But for now I just wanted to have some output from amdctl -g, which is as below:

witek@sirius:~/dev/sources/cpp/amdctl$ sudo /home/witek/dev/sources/cpp/amdctl/amdctl -g
DEBUG: Checking CPU info.
DEBUG: Setting variables based on CPU model.
AMD14H not fully supported. Test mode will be automatically enabled!
Voltage ID encodings: SVI (serial)
Detected CPU model 2h, from family 14h with 2 CPU cores.
DEBUG: Getting data from CPU 0 at register c0010061
DEBUG: Getting data from CPU 0 at register c0010063

Core 0 | P-State Limits (non-turbo): Highest: 1 ; Lowest 3 | Current P-State: 3
 Pstate Status CpuFid CpuDid CpuVid CpuMult CpuFreq CpuVolt IddVal IddDiv CpuCurr CpuPower
DEBUG: Getting data from CPU 0 at register c0010064
      0      1     16     16     16   0.00x    0MHz  1350mV     45     10   4.50A    6.08W
DEBUG: Getting data from CPU 0 at register c0010065
      1      1     18     18     27   0.00x    0MHz  1212mV     30     10   3.00A    3.64W
DEBUG: Getting data from CPU 0 at register c0010066
      2      1     48     48     47   0.00x    0MHz   962mV    151    100   1.51A    1.45W
DEBUG: Getting data from CPU 0 at register c0010067
      3      0      0      0      0   0.00x    0MHz  1550mV      0      1   0.00A    0.00W
currentDEBUG: Getting data from CPU 0 at register c0010071
      1     48     48     16   0.00x    0MHz  1350mV
DEBUG: Getting data from CPU 1 at register c0010061
DEBUG: Getting data from CPU 1 at register c0010063

Core 1 | P-State Limits (non-turbo): Highest: 1 ; Lowest 3 | Current P-State: 1
 Pstate Status CpuFid CpuDid CpuVid CpuMult CpuFreq CpuVolt IddVal IddDiv CpuCurr CpuPower
DEBUG: Getting data from CPU 1 at register c0010064
      0      1     16     16     16   0.00x    0MHz  1350mV     45     10   4.50A    6.08W
DEBUG: Getting data from CPU 1 at register c0010065
      1      1     18     18     27   0.00x    0MHz  1212mV     30     10   3.00A    3.64W
DEBUG: Getting data from CPU 1 at register c0010066
      2      1     48     48     47   0.00x    0MHz   962mV    151    100   1.51A    1.45W
DEBUG: Getting data from CPU 1 at register c0010067
      3      0      0      0      0   0.00x    0MHz  1550mV      0      1   0.00A    0.00W
currentDEBUG: Getting data from CPU 1 at register c0010071
      1     16     16     16   0.00x    0MHz  1350mV
Northbridge:

The data may be not correcty calculated, like for example the clock divider or the cpu frequency. It looks like there is something wrong in my case in the number of currently P-State. Consider this:

Core 0 | P-State Limits (non-turbo): Highest: 1 ; Lowest 3 | Current P-State: 3
Core 1 | P-State Limits (non-turbo): Highest: 1 ; Lowest 3 | Current P-State: 1

it says that cores are in a different current P-States, however this:

currentDEBUG: Getting data from CPU 0 at register c0010071
      1     48     48     16   0.00x    0MHz  1350mV
currentDEBUG: Getting data from CPU 1 at register c0010071
      1     16     16     16   0.00x    0MHz  1350mV

may indicate (by the 1350mV voltage) that both cores are in the first P-State, so each core should consume more than 6W. This seems to be correct because my power/energy meter shows that the computer consumes around 14W.

@kevinlekiller, before I continue with my research I have a question: if I totaly disable one core of the CPU, will the total consumed power drop about 6W? Then instead of 14W of consumption, my computer will draw around 8W?

Second thing: I have also used a cpupower utility to control the frequency of CPU but it looks like it doesn't control the power states as my computer was still at 14W no matter what I have chosen. Third thing: I have also put one core into an offline state by executing as root: echo 0 > /sys/devices/system/cpu/cpu1/online but that not really helped if it goes about power consumption: it still draw 14W, however the htop command line utility showed me only one active CPU core available.

@kevinlekiller, could you please help me a bit, to understand?

wmarkow commented 5 years ago

It looks like CPU_FID can be read from MainPllOpFreqId (bits 5:0) of D18F3xD4 register. How to read from D18F3xD4 register?

wmarkow commented 5 years ago

I know how to read D18F3xD4 from linux command line. We need to read from a PCI device. To get the list of PCI devices:

witek@sirius:~$ lspci
00:00.0 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 14h Processor Root Complex
00:01.0 VGA compatible controller: Advanced Micro Devices, Inc. [AMD/ATI] Wrestler [Radeon HD 6320]
00:01.1 Audio device: Advanced Micro Devices, Inc. [AMD/ATI] Wrestler HDMI Audio
00:11.0 SATA controller: Advanced Micro Devices, Inc. [AMD/ATI] SB7x0/SB8x0/SB9x0 SATA Controller [IDE mode] (rev 40)
00:12.0 USB controller: Advanced Micro Devices, Inc. [AMD/ATI] SB7x0/SB8x0/SB9x0 USB OHCI0 Controller
00:12.2 USB controller: Advanced Micro Devices, Inc. [AMD/ATI] SB7x0/SB8x0/SB9x0 USB EHCI Controller
00:14.0 SMBus: Advanced Micro Devices, Inc. [AMD/ATI] SBx00 SMBus Controller (rev 42)
00:14.2 Audio device: Advanced Micro Devices, Inc. [AMD/ATI] SBx00 Azalia (Intel HDA) (rev 40)
00:14.3 ISA bridge: Advanced Micro Devices, Inc. [AMD/ATI] SB7x0/SB8x0/SB9x0 LPC host controller (rev 40)
00:14.4 PCI bridge: Advanced Micro Devices, Inc. [AMD/ATI] SBx00 PCI to PCI Bridge (rev 40)
00:15.0 PCI bridge: Advanced Micro Devices, Inc. [AMD/ATI] SB700/SB800/SB900 PCI to PCI bridge (PCIE port 0)
00:15.2 PCI bridge: Advanced Micro Devices, Inc. [AMD/ATI] SB900 PCI to PCI bridge (PCIE port 2)
00:15.3 PCI bridge: Advanced Micro Devices, Inc. [AMD/ATI] SB900 PCI to PCI bridge (PCIE port 3)
00:18.0 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 12h/14h Processor Function 0 (rev 43)
00:18.1 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 12h/14h Processor Function 1
00:18.2 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 12h/14h Processor Function 2
00:18.3 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 12h/14h Processor Function 3
00:18.4 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 12h/14h Processor Function 4
00:18.5 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 12h/14h Processor Function 6
00:18.6 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 12h/14h Processor Function 5
00:18.7 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 12h/14h Processor Function 7
03:00.0 Ethernet controller: Broadcom Limited NetLink BCM57781 Gigabit Ethernet PCIe (rev 10)
04:00.0 USB controller: Texas Instruments TUSB73x0 SuperSpeed USB 3.0 xHCI Host Controller (rev 02)

Notice that 00:18.3 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 12h/14h Processor Function 3 is on the list.

To read the D18F3xD4 register:

witek@sirius:~$ sudo setpci -s 00:18.3 D4.L
00024f51

You need to execute it as superuser, otherwise the result will be like ffffffff.

In my case the FID are the bits 5:0. D18F3xD4 = 0x00024f51 = 0b100100111101010001, so MainPllOpFreqId = 0b010001 = 0x11 = 17

The https://www.amd.com/system/files/TechDocs/43170_14h_Mod_00h-0Fh_BKDG.pdf (page 296) gives the formula for a PLL COF:

The main PLL COF = 100 MHz * (D18F3xD4[MainPllOpFreqId] + 10h).

which for my case is 100MHz (11h + 10h) = 100MHz 21h = 100MHz * 33 = 3300 MHz

wmarkow commented 5 years ago

In amdctl.c use

void getAddr(const char * loc, const uint32_t reg)

to read the value from PCI device. To read MainPllOpFreqId use this:

getAddr("18.3", 0xD4); // result will be stored in buffer variable
int MainPllOpFreqId = getDec("5:0");// reads the bits range from buffer variable

kevinlekiller commented 5 years ago

I'll have a look at your issue tonight, haven't had time yet, sorry.

kevinlekiller commented 5 years ago

if I totaly disable one core of the CPU, will the total consumed power drop about 6W? Then instead of 14W of consumption, my computer will draw around 8W?

I'm not sure if disabling 1 core reduces power usage (on 14h CPU's), maybe disabling the core just tells the kernel to not use it ? I know on some CPU's, the same voltage is applied to all cores, which might be why when you disable a core you see the same power usage?

The data may be not correcty calculated, like for example the clock divider or the cpu frequency. It looks like there is something wrong in my case in the number of currently P-State.

According to the manual you linked, there should be 8 pstates (page 55).

From what I can gather, the cpu frequency for your cpu needs to be calculated like this (page 430):

getReg(0xc0010064); to get data from p-state 0

double cpuDivisor = getDec("8:4") + (getDec("3:0") * 0.25) + 1;

Then you'd get the COF, like you did in https://github.com/kevinlekiller/amdctl/issues/16#issuecomment-507638054

getAddr("18.3", 0xD4);

double COF = getDec("5:0") / cpuDivisor;

Then you can calculate cpu frequency with double cpuFreq = COF * cpuDivisor;

You'd need to edit amdctl.c to check if the family is 14h and do this in the loop here: https://github.com/kevinlekiller/amdctl/blob/master/amdctl.c#L238 since it seems to be implied in the manual that COF is volatile.

They also say this, which I don't understand :Only the following core clock divisors can be created by CpuDidMSD and CpuDidLSD:• 1.0-15.75 in 0.25 steps• 16.0-26.5 in 0.50 steps (CpuDidLSD[0] has no effect in this range), maybe you can make sense of it ?

wmarkow commented 5 years ago

Thanks @kevinlekiller for the tips. I have modified the amdctl.c so at least it prints the P-States table for 14H family.

Related to this:

I'm not sure if disabling 1 core reduces power usage (on 14h CPU's), maybe disabling the core just tells the kernel to not use it ? I know on some CPU's, the same voltage is applied to all cores, which might be why when you disable a core you see the same power usage?

I did more tests in this topic. I have HP T610 Plus thin client with AMD G-T56N processor (with two cores). OS is latest Debian without GUI (just a console output is available). The processor has only three valid P-States available. I have no idea who configured them, maybe it is by factory default or maybe Debian manages this. In debian I have also cpupower available with an automatic power management set to on demand. It means that Debian controls the P-States automatically, it depends on the current load.

I did the following tests:

after booting the computer, amdctl shows that both cpus are in P-State 2, so it is low power mode where one core consumes around 1.5W of energy, so both cores draws 3W. Computer does "nothing". It is just a pure Debian. htop utility shows 0% CPU usage. My power meter shows 14W of power consumption. 3W are taken by CPUs, so 11W is consumed by something else (like motherboard, graphic, peripherals, etc.)
I run stress-ng -c 4 so the computer will do some hard work. CPU usage goest to 100% on both CPUs. After 5 minutes power meter shows 22W. According to amdctl my current P-State is 0, which means high performance. In this state one CPU core draws around 6W, so 12W by two cores. Let's check the math: 11W for peripherals + 12W for CPUS = 23W. It matches, my power metersh showed 22W.
I haved killed the stress-ng -c 4 and waited a small amount of time. Power meter goes back to 14W, CPU usage drops to 0%, current P-State is 2, so it consume less energy.
I have disabled one core (cpu1) with command echo 0 > /sys/devices/system/cpu/cpu1/online Since one core is off, I hoped that the power usage will drop by 1.5W (this is the power drawed by ine core in power saving mode P-State 2). However my energy meter showed still 14W. Looks like the disabled core still takes 1.5W. Interesting. I can't explaint it. I have run stress-ng -c 4 again, so computer can do some hard work but this time only one core is available. CPU usage goest to 100%, current P-State is 0 (high performance) but only one core is available. Power consumption should be : 11W for peripherals + 1.5W for disabled core + 6W for other core = 18.5W. However the power meter showed 17W. There is a mismatch around 1.5W. I do not know how to explain this. In general it proofs that a disabled core doesn't take much power, at least in my machine.

Finally, it looks like Debian with cpupower automatically manages the power consumption. The power management policy switches the low power mode by default and switches to high performance automatically on demand. I think I will stay with this policy, so for now I will not use amdctl to controll P-States, however I will use it to monitor it as it has a nice output table with everything (cpu clock, voltage and power).

wmarkow commented 5 years ago

They also say this, which I don't understand :Only the following core clock divisors can be created by CpuDidMSD and CpuDidLSD:• 1.0-15.75 in 0.25 steps• 16.0-26.5 in 0.50 steps (CpuDidLSD[0] has no effect in this range), maybe you can make sense of it ?

I have no idea. Maybe: there is 5 bits for a decimal value and 4 bits for fraction of the divider. When all bits are set then the equation:

double cpuDivisor = getDec("8:4") + (getDec("3:0") * 0.25) + 1;

gives the result 35.75. Maybe not all avlaues are supported by the processor, however the equation can deliver them?

kevinlekiller commented 5 years ago

Since one core is off, I hoped that the power usage will drop by 1.5W (this is the power drawed by ine core in power saving mode P-State 2). However my energy meter showed still 14W. Looks like the disabled core still takes 1.5W. Interesting. I can't explaint it.

If you change the P-State (2) of the disabled core to have lower clock speed / voltage, does that reduce power usage ?

wmarkow commented 5 years ago

I do not know if that is possible to do. I can only disable second core cpu1. When it is disabled then:

/proc/cpiunfo shows only one core
/dev/cpu/1/msr is missing
amdctl shows information about core 0, so it looks like I can't manipulate with second core, because it is disabled.

Moreover I have modified amdctl to only read from 14H family; write operation is not supported.

vinibali commented 3 years ago

Hi! I have a facelifted Brazos motherboard with a AMD E1-1500, which most of the time stays cool. Did you managed to figure out what should be changed? TPC seems to handle the P-States correctly

kevinlekiller commented 2 years ago

This now supported in #38

kevinlekiller / amdctl

Help on family 14H #16