search

cyring / CoreFreq

CoreFreq : CPU monitoring and tuning software designed for 64-bit processors.
https://www.cyring.fr
GNU General Public License v2.0
1.98k stars 126 forks source link

RAPL for Intel/AMD architectures #57

Closed cyring closed 4 years ago

cyring commented 6 years ago

Replace PWR_ACCU_SandyBridge with PWR_ACCU_Skylake

#define PWR_ACCU_Skylake(Pkg, T)                                        \
({                                                                      \
        RDCOUNTER(Pkg->Counter[T].Power.ACCU[PWR_DOMAIN(PKG)],          \
                                                MSR_PKG_ENERGY_STATUS); \
                                                                        \
        RDCOUNTER(Pkg->Counter[T].Power.ACCU[PWR_DOMAIN(CORES)],        \
                                                MSR_PP0_ENERGY_STATUS); \
                                                                        \
        RDCOUNTER(Pkg->Counter[T].Power.ACCU[PWR_DOMAIN(UNCORE)],       \
                                                MSR_PP1_ENERGY_STATUS); \
                                                                        \
        RDCOUNTER(Pkg->Counter[T].Power.ACCU[PWR_DOMAIN(RAM)],          \
                                                MSR_DRAM_ENERGY_STATUS);\
})
cyring commented 6 years ago

KBL RAPL and Voltage during high load ... corefreq_kbl_rapl_high ... during low load corefreq_kbl_rapl_low

cyring commented 6 years ago

Measurements issue with i7-3770 (IvyBridge). TDP specifications have to be 77 W 2018-07-30-175649_644x364_scrot 2018-07-30-175557_644x364_scrot 2018-07-30-175513_644x364_scrot

cyring commented 6 years ago

Skylake: add missing code to measure DRAM power https://github.com/cyring/CoreFreq/blob/6712f73d42313d07f85659eef70cc8dff95797b2/corefreqk.c#L6072

            Delta_PWR_ACCU(Proc, RAM);

https://github.com/cyring/CoreFreq/blob/6712f73d42313d07f85659eef70cc8dff95797b2/corefreqk.c#L6090

            Save_PWR_ACCU(Proc, RAM);

2018-08-03-171023_644x316_scrot 2018-08-03-171110_644x316_scrot 2018-08-03-171214_644x316_scrot

cyring commented 6 years ago

Workaround for IVB [Intel(R) Core(TM) i7-3770 CPU @ 3.40GHz] https://github.com/cyring/CoreFreq/blob/68db124338be91b5a97a08cab7830cfbb24e472d/corefreqd.c#L443

    Shm->Proc.Power.Unit.Watts = Proc->PowerThermal.Unit.PU > 0 ?
            1.0 / (double) (1 << Proc->PowerThermal.Unit.PU) : 0;

    Shm->Proc.Power.Unit.Watts /= (Proc->CPU.Count >> Proc->Features.HTT_Enable);

2018-09-13-120420_644x316_scrot Accurate ?

cyring commented 5 years ago

Issue closed until other hardware is available for testings.

cyring commented 5 years ago
Architecture Processor ESU (J)1 Pkg load (J)2 Cores load (J)2 TDP (W)3
Skylake/S i5-6600K 0.000061035 84.761535645 75.259704590 91
Skylake/S i7-6700 0.000061035 55.686950684 44.473632812 65
Haswell/U i3-4010U 0.000061035 6.958801270 4.341308594 15
Haswell/U i7-4650U 0.000061035 18.376159668 14.542602539 15
IvyBridge/EP E5-1607 0.000015259 34.55101032 26.025909424 130
SandyBridge i7-2710QE 0.000015259 44.356430054 40.928970337 45
Architecture Processor ESU (J) Pkg load (J) Cores load (J) TDP (W)
Pinnacle Ridge 2700X 0.000015259 115.171279907 116.535186768 105

Remarks

  1. RAPL Units Register.
  2. All Cores fully loaded One second interval RAPL Energy architectural counter
  3. Manufacturer specification
cyring commented 5 years ago

Proposal for the Power & Voltage view

AMD Zen

# Freq MHz VID Vcore Energy(J) Counter(Raw)
#0 4015.73 . . Package 131.852859497 9876543210
#1 4015.67 . . Cores 122.611679077 .
#2 4015.74 54 1.2125 Uncore 10.000000000 9876543210
#3 4015.73 . . Memory 5.000000000 .
# Freq MHz VID Vcore Power(W) Core(W)
#0 4015.73 . . Package 131.852859497 13.000000000
#1 4015.67 . . Cores 122.611679077 .
#2 4015.74 54 1.2125 Uncore 10.000000000 11.000000000
#3 4015.73 . . Memory 5.000000000 .

Intel

# Freq MHz VID Vcore Energy(J) Counter(Raw)
#0 4015.73 . . Package 131.852859497 876543210987654321
#1 4015.67 . . Cores 122.611679077 876543210987654321
#2 4015.74 54 1.2125 Uncore 10.000000000 106543210987654321
#3 4015.73 . . Memory 5.000000000 056543210987654321
# Freq MHz VID Vcore Power(W) Core(W)
#0 4015.73 . . Package 131.852859497 .
#1 4015.67 . . Cores 122.611679077 .
#2 4015.74 54 1.2125 Uncore 10.000000000 .
#3 4015.73 . . Memory 5.000000000 .

Remarks

cyring commented 5 years ago

RAPL in the AMD Zen architecture

Energy consumed

Ryzen 3xxx

61916877-a044b500-af07-11e9-8404-d2a39c56fe59

Ryzen 2xxx

61589073-3e84f380-ab73-11e9-9016-573618e4daa1

Topology

Ryzen 2xxx

Processor                              [AMD Ryzen 7 2700X Eight-Core Processor ]
|- Architecture                                            [Zen+ Pinnacle Ridge]
|- Vendor ID                                                      [AuthenticAMD]
|- Microcode                                                        [ 134251019]
|- Signature                                                            [ 8F_08]
|- Stepping                                                             [     2]
|- Online CPU                                                           [ 16/16]
...
Power & Thermal Monitoring:                                                     
...
|- Units                                                                        
   |- Power                                               watt   [  0.125000000]
   |- Energy                                             joule   [  0.000015259]
   |- Window                                            second   [  0.000976562]

CoreFreq_2700X_Topology

Ryzen 3xxx

Processor                                  [AMD Ryzen 7 3700X 8-Core Processor ]
|- Architecture                                               [Zen/Summit Ridge]
|- Vendor ID                                                      [AuthenticAMD]
|- Microcode                                                        [ 141561873]
|- Signature                                                            [ 8F_71]
|- Stepping                                                             [     0]
|- Online CPU                                                           [ 16/16]
...
Power & Thermal                                                                 
...
|- Units                                                                        
   |- Power                                               watt   [  0.125000000]
   |- Energy                                             joule   [  0.000015259]
   |- Window                                            second   [  0.000976562]

CPU Pkg  Apic  Core Thread  Caches      (w)rite-Back (i)nclusive              
 #   ID   ID    ID     ID  L1-Inst Way  L1-Data Way      L2  Way      L3  Way 
00: BSP     0     0      0      32  8        32  8       512  8     32768  9  
01:   0     2     1      0      32  8        32  8       512  8     32768  9  
02:   0     4     2      0      32  8        32  8       512  8     32768  9  
03:   0     6     3      0      32  8        32  8       512  8     32768  9  
04:   0     8     4      0      32  8        32  8       512  8     32768  9  
05:   0    10     5      0      32  8        32  8       512  8     32768  9  
06:   0    12     6      0      32  8        32  8       512  8     32768  9  
07:   0    14     7      0      32  8        32  8       512  8     32768  9  
08:   0     1     0      1      32  8        32  8       512  8     32768  9  
09:   0     3     1      1      32  8        32  8       512  8     32768  9  
10:   0     5     2      1      32  8        32  8       512  8     32768  9  
11:   0     7     3      1      32  8        32  8       512  8     32768  9  
12:   0     9     4      1      32  8        32  8       512  8     32768  9  
13:   0    11     5      1      32  8        32  8       512  8     32768  9  
14:   0    13     6      1      32  8        32  8       512  8     32768  9  
15:   0    15     7      1      32  8        32  8       512  8     32768  9

Threadripper 2950X

Remark: Threadripper results below are from an old CoreFreq version

Processor                      [AMD Ryzen Threadripper 2950X 16-Core Processor ]
|- Architecture                                                    [Zen+ Colfax]
|- Vendor ID                                                      [AuthenticAMD]
|- Microcode                                                        [ 134251019]
|- Signature                                                            [ 8F_08]
|- Stepping                                                             [     2]
|- Online CPU                                                           [ 32/32]
...
Power & Thermal                                                                 
...
|- Units                                                                        
   |- Power                                               watt   [  0.007812500]
   |- Energy                                             joule   [  0.000000954]
   |- Window                                            second   [  0.000976562]

CPU Pkg  Apic  Core Thread  Caches      (w)rite-Back (i)nclusive              
 #   ID   ID    ID     ID  L1-Inst Way  L1-Data Way      L2  Way      L3  Way 
00: BSP     0     0      0      64  4        32  8       512  8     32768 10  
01:   0     2     1      0      64  4        32  8       512  8     32768 10  
02:   0     4     2      0      64  4        32  8       512  8     32768 10  
03:   0     6     3      0      64  4        32  8       512  8     32768 10  
04:   0     8     4      0      64  4        32  8       512  8     32768 10  
05:   0    10     5      0      64  4        32  8       512  8     32768 10  
06:   0    12     6      0      64  4        32  8       512  8     32768 10  
07:   0    14     7      0      64  4        32  8       512  8     32768 10  
08:   1    16     0      0      64  4        32  8       512  8     32768 10  
09:   1    18     1      0      64  4        32  8       512  8     32768 10  
10:   1    20     2      0      64  4        32  8       512  8     32768 10  
11:   1    22     3      0      64  4        32  8       512  8     32768 10  
12:   1    24     4      0      64  4        32  8       512  8     32768 10  
13:   1    26     5      0      64  4        32  8       512  8     32768 10  
14:   1    28     6      0      64  4        32  8       512  8     32768 10  
15:   1    30     7      0      64  4        32  8       512  8     32768 10  
16:   0     1     0      1      64  4        32  8       512  8     32768 10  
17:   0     3     1      1      64  4        32  8       512  8     32768 10  
18:   0     5     2      1      64  4        32  8       512  8     32768 10  
19:   0     7     3      1      64  4        32  8       512  8     32768 10  
20:   0     9     4      1      64  4        32  8       512  8     32768 10  
21:   0    11     5      1      64  4        32  8       512  8     32768 10  
22:   0    13     6      1      64  4        32  8       512  8     32768 10  
23:   0    15     7      1      64  4        32  8       512  8     32768 10  
24:   1    17     0      1      64  4        32  8       512  8     32768 10  
25:   1    19     1      1      64  4        32  8       512  8     32768 10  
26:   1    21     2      1      64  4        32  8       512  8     32768 10  
27:   1    23     3      1      64  4        32  8       512  8     32768 10  
28:   1    25     4      1      64  4        32  8       512  8     32768 10  
29:   1    27     5      1      64  4        32  8       512  8     32768 10  
30:   1    29     6      1      64  4        32  8       512  8     32768 10  
31:   1    31     7      1      64  4        32  8       512  8     32768 10

Issue

AMD specifications

MSRC001_029B [Package Energy Status] (Core::X86::Msr::PKG_ENERGY_STAT)
Read-only,Volatile. Reset: 0000_0000_0000_0000h. 
_lthree[1:0]; MSRC001_029B
Bits Description
63:32 Reserved. 
31:0 TotalEnergyConsumed.
CCX: Core Complex where more than one core shares L3 resources.

CPUID_Fn8000001E_EBX [Core Identifiers] (Core::X86::Cpuid::CoreId) Read-only. See Core::X86::Cpuid::ExtApicId. _lthree[1:0]_core[3:0]_thread[1:0]; CPUID_Fn8000001E_EBX Bits Description 31:16 Reserved. 15:8 ThreadsPerCore: threads per core. Read-only. Reset: XXh. The number of threads per core is ThreadsPerCore+1. 7:0 CoreId: core ID. Read-only. Reset: Fixed,XXh. Description: For Family 17, Model 1, Revision 1 and later: CoreId = ({2'b0, DieId[1:0], LogicalComplexId[0], LogicalThreadId[2:0]} >> SMT).

CPUID_Fn8000001E_ECX [Node Identifiers] (Core::X86::Cpuid::NodeId) Read-only. _lthree[1:0]_core[3:0]_thread[1:0]; CPUID_Fn8000001E_ECX Bits Description 31:11 Reserved. 10:8 NodesPerProcessor: Node per processor. Read-only. Reset: XXXb. ValidValues: Value Description 0h 1 node per processor. 1h 2 nodes per processor. 2h Reserved. 3h 4 nodes per processor. 7h-4h Reserved. 7:0 NodeId: Node ID. Read-only. Reset: Fixed,XXh. Description: For Family 17, Model 1, Revision 1 and later: {5'b00000,1'b[SOCKET_ID],2'b[DIE_ID]}.


###  Improvements
* find the CCX identifier into the CPU topology
* read the Package energy counter per CCX
* sum the per CCX values into a Package value

### Questions
* what about the multi dies processors (TR, Naples, Rome)
cyring commented 5 years ago

Core Complex ID

Ryzen 3xxx [SMT ON]

CPU# Pkg ID Apic Ext_ID Core ID Thread ID Apic[3:0] CCX ID1 _lthree scope2
00 BSP 0 0 0 0000 0 Y
01 0 2 1 0 0010 0 .
02 0 4 2 0 0100 0 .
03 0 6 3 0 0110 0 .
04 0 8 4 0 1000 1 Y
05 0 10 5 0 1010 1 .
06 0 12 6 0 1100 1 .
07 0 14 7 0 1110 1 .
08 0 1 0 1 0001 0 .
09 0 3 1 1 0011 0 .
10 0 5 2 1 0101 0 .
11 0 7 3 1 0111 0 .
12 0 9 4 1 1001 1 .
13 0 11 5 1 1011 1 .
14 0 13 6 1 1101 1 .
15 0 15 7 1 1111 1 .
  1. ApicId[3] = Logical CCX L3 complex ID 
    CCX_ID = ( leaf8000001e.EAX.ExtApicId & 0b1000 ) >> 1
  2. First ID of the CCX instance
cyring commented 5 years ago

After long hours of the 3700X debugging, it appears that the RAPL Package Energy Counter delta remains the same, whatever the CPU is used to read the msr CoreFreq_Ryzen_3700X_Energy

Edit: as specified, the RAPL Energy status is package scope which returns the same value whatever is the Core used for reading. CCX is not involved in the issue.

_Call for help on Reddit_

cyring commented 5 years ago
olejon commented 5 years ago

Here are my Screenshots + Output as requested in #129

cyring commented 5 years ago

To all reading this issue, look in the above screenshots for the Package Power measurements; Cores all stressed :

Both cases, same algorithm based on RAPL registers; what can we conclude ?

olejon commented 5 years ago

IMPORTANT NOTE:

olejon commented 5 years ago

EDIT: Some additional info... Please read IMPORTANT NOTE section for X370/X470, especially the first.

olejon commented 5 years ago

What I wrote very much confirmed:

CoreFreq manages to get it to at least more than this, using "Conic Compute" IIRC, even maybe 82-84 C IIRC

cyring commented 5 years ago

Is the B450 Chipset supported ? Because we also have the Package Power issue above

cyring commented 5 years ago
Processor Chipset Pkg Pwr
3700X X370 KO
3700X B450 KO
3600X X570 OK
cyring commented 5 years ago

Conic is an old friend of the 486DX age https://github.com/cyring/CoreFreq/blob/e21a6d5480fff6a095edec7b6ea01310ad25c0e9/corefreqm.c#L196

cyring commented 5 years ago

AMD64 Architecture-Programmer’s Manual-Volume 2: System Programming

Hardware Performance Monitoring and Control

Software Algorithm
The following algorithm should be used to calculate the average power consumed by a processor core 
during the measurement interval TM. To obtain a stable average power value, TM should be on the 
order of several milliseconds.
• Determine the value of the ratio of Tsample to the TSC period (CpuPwrSampleTimeRatio) by 
executing CPUID Fn8000_0007. Call this value N.
N = CPUID Fn8000_0007_ECX[31:0].
• Read the full range of the cumulative energy value from the register MaxCpuSwPwrAcc. 
Jmax = value returned from RDMSR MaxCpuSwPwrAcc.
• At time x, read CpuSwPwrAcc and the TSC
Jx = value returned by RDMSR CpuSwPwrAcc 
Tx = value returned by RDTSC
• At time y, read CpuSwPwrAcc and the TSC again
Jy = value returned by RDMSR CpuSwPwrAcc
Ty = value returned by RDTSC
Calculate the average power consumption for the processor core over the measurement interval TM = 
(Ty – Tx). 
• If (Jy < Jx), rollover has occurred; set Jdelta = (Jy + Jmax) – Jx
else Jdelta = Jy – Jx
• PwrCPUave = N * Jdelta / (Ty - Tx) 
Units of result is milliwatts.
olejon commented 5 years ago

I think we need results from the 3700X + X570 combinaison.

That will work just like 3600X.

cyring commented 5 years ago

That will work just like 3600X

I think the same, but the table won't be completed without such combination. Does the issue happen because of the chipset or the Processor model...

olejon commented 5 years ago

Well you need people with other/more HW then, or make it easy to PayPal donate to you so you can buy CPUs. MBs matter little for CoreFreq, so those you can get very cheap.

I can't say anything before getting my 3950X, which is Ryzen 9, but if works like 3600X, why should Ryzen 7 don't?

I'll pre-order as soon as available here. If I'm quick maybe I'll get it in the end of September or beginning of October (depending on when AMD actually launches it, but says already on amd.com September), and if that CPU works exactly the same in CoreFreq, why wouldn't the 3700X?

IDK why they call all CPUs up to 3700X Ryzen 5, then 3700X up to 3900X Ryzen 7 and 3900X+ Ryzen 9. The Ryzen 5 is what makes no sense for me, for a 3600X CPU. Shouldn't it then be Ryzen 6? And 3800X be Ryzen 8?

Very few should buy the 3700X and no one the 3800X, after some reading. If for gaming? Buy the 3600 (without X), which doesn't bottleneck any modern GPU, at least not on X570, but I've seen similar results on X470 which is a cheaper option (and both that CPU and those older Gen MBs will lower in price).

At first considered the 3700X. AMD called it the competition to the latest i7-8700K I think, which is a darn good CPU so. But many said that they should call it more a direct competition to the latest i9, although the i9 does better in benchs, it's mostly because of the 5 GHz Turbo. But no real world usage difference, that probably goes in favor of AMD.

AMD said the 3900X was the direct competitor to the i9-9900K. Here the 3900X cost €60 more. Those prices will drop though - they were just released. Nobody has it in stock. It was but super demand.

And AMD's Turbo out of the box is SMT which equals to Sync All Cores which even in Auto mode, meaning all Turbo to max stock Turbo and not above that, is a pretty serious OC on Intel, letting all cores Turbo makes it MUCH hotter in my experience, while not noticable on AMD. Requires good cooling, and I imagine especially since it has Hyperthreading, which my latest didn't.

With a good cooler the 3700X/3800X can probably get a decent bump in Turbo GHz, maybe not 5 GHz but beat the i9 anyways.

At least maybe it can compete with the i9-9900 it (a CPU I never see mentioned in videos), but both are Gen "Coffee Lake Refresh". When I checked my Norwegian online store, it seemed a better choice as cheaper with a fan included (doesn't say which, just that the product is "In Intel Box", but so does the K, but fan the K does never have, just an overly fancy box that people give it lower OR higher rating for, lower because really after opening it's just a waste of carton. Non-K being about just €2 cheaper though.

Differences:

Similarities:

cyring commented 5 years ago

EPYC Rome 64C/128T in a 2 sockets configuration is so far my dream setup. But true, any Ryzen should be enough to study the architecture and those RAPL registers....

olejon commented 5 years ago

You mean this?

AMD EPYC 7742 64-Core 2.25GHz (3.4 GHz Max Boost) Socket SP3 225W 100-100000053WOF Server Processor

https://www.newegg.com/p/N82E16819113581

$7,219.99 (can't promise such a donation...)

And that's just the price for the CPU :-O

I count 6 Authorized Distributors & Online Retailers from amd.com :P

Try it in the cloud first at least just for a short time? :P Can run bare metal there I assume, and for much less.

olejon commented 5 years ago

O-la-la:

A screenshot of popular benchmarking tool Geekbench revealed by regular Twitter leaker APISAK (@TUM_APISAK) supposedly shows AMD's new 16-core Ryzen 9 3950X beating Intel's current desktop flagship – the Core i9-9980XE. The latter has 18 cores and costs $2000 while AMD’s new king of the hill costs just $749 and has two fewer cores.

Image

Think this is worth it. We'll see Norwegian highest cost of living in the world price. Almost doubles my Geekbench score from 3600X, but this is probably without any OC except SMT which seems to be on on all MBs.

cyring commented 5 years ago

(just kidding) Should we report to them that we found some RAPL Package counter - chipset issues before they launch such a monster processor ?

cyring commented 5 years ago
Processor Chipset Pkg Pwr
3700X X370 KO
3700X B450 KO
3600X X570 OK
3700X X570 OK

IMG_20190814_012329

olejon commented 5 years ago

Sorry I didn't understand this. If you were really kidding. Remember I'm not a CPU expert, and don't have enough RAM in my brain for more.

Should we report to them that we found some RAPL Package counter - chipset issues before they launch such a monster processor ?

Yes, as said the 3700X would surely work similar in CoreFreq on X570 since 3600X does. I wouldn't care about X370 as AMD doesn't recommend it. Though you should really try to get or loan an X470 and see if it works on that (after all BIOS updates applied). That is what budget users will go for, and likely the 3600 (without X) as IIRC AMD and YT videos recommend TDP = 65W and not more. It'll surely work with the 3700X as well, and if so I assume the 3600 shows correct data. I think the recommendation of TDP not higher than 65W is because of better OC capabilities.

Also X570 has a new standardized motherboard design all manufacturers follow. Like the PCH fan, VRAM cooling and these things work together for cooling the chips. May be the reason of the 65W recommendation as well on X470. I don't think AMD is lying here, since they DO want to sell their new CPUs, even if you use an X470. They have en entire program for compatibility after all (BIOS updates sent to all manufacturers, before launch, telling both them and sellers to update the MBs before selling to make sure a 3rd Gen Ryzen CPU actually works).

I'll have some more screenshots and Power Usage comparisons Linux vs Windows. Managed to improve both using AMD supported OC (not much). Linux improved more than Windows, but both improved.

In what PCs is B450 used?

cyring commented 5 years ago

I do not know the PC which hosts the B450. I just report the strings the kernel has probe through its DMI implementation.

Whatever I won't go further into this Ryzen RAPL issue, X570 is the way to go.

(BTW plz let me know if you have a chance to test the modified code in the issue #140 )

Regards

olejon commented 5 years ago

I think I see. MBs, most mATX or mITX. Looking at the "MSI B450I GAMING PLUS AC" and the "Gigabyte B450 I AORUS PRO WIFI" (latter having both Intel for Wi-Fi/Bluetooth and Ethernet, former Realtek...) both mITX they're pretty impressive considering the price, both have 802.11ac Wi-Fi, and seems to have 2 antenna connectors, antennas don't seem to be included. People write positive things about all that have reviews. About the mITX that they're perfect for a PC in the bookshelf and such, and that Wi-Fi coverage is just "fine" (probably 2 antennas would make that much better).

The most expensive is ATX, "ASUS ROG STRIX B450-E GAMING" and comes with antennas. And being the most expensive it's still only €189 in Norway. And the first with Wi-Fi just €129.

They haven't updated the site that they support 3rd Gen Ryzen, but looked at the cheapest ASUS one (mATX) to see CPU support for it, and the mentioned ATX above, the one I would have bought since the sister I suppose to my PRIME-PRO, although it's not a PRO, and called "PRIME B450M-A". Both say they support even the 3900X so... Surely the 3950X when coming.

Of course lacking is PCIe4 or 3200 MHz RAM without (OC) behind it, but several has 2 x M.2 and says they support RAID 0. The MSI ones have most reviews.

MSI ones also brags about "2 x Turbo M.2 with up to 32 GB/s per unit using PCIe3 x4 and support for RAID 0" (so per unit, with RAID 0 it can give like 64 GB/s? Ehm fastest M.2 I can find is a Premium 2T actually packaged for a PCIe slot and has about 7 GB/s and only supports Windows Server on the Windows front, but also says "Linux 3.3 and newer and FreeBSD" haha.

Surely other stores have faster ones but 32 GB/s? Marketing? What the heck does Turbo M.2 even mean? It's just PCIe3 on all and that'll be the limit. Also don't like how some B450's brag about supporting having an SSD and an HDD and making sure often used files are on the SSD and less used files on, yeah, you get it. Like the complete disaster Apple's Hybrid drives were. Speeds usually HDD-ish. And how's the setup? UEFI or probably require some Windows software.

Seems to be pretty RGB heavy built-in and connectors for more, all of them, kind of weird for a small MB that's not usually put for display? IDK, maybe they are. I've never built with anything else than ATX so.

Want space, and my Chassis is the Graphite Series 760T Arctic White Full-Tower Windowed Case (see extensive gallery). The "red" is really nice if you want all black, just you must change the 3 chassis fans it comes with then because they have yeah, red LEDs, while mine has white. And mix of "artic white" color with black and black tinted windows makes it fit perfectly my room. Chassis' fans aren't bad either.

With the Ryzen without custom OC can really set in UEFI "Allow fan stop" for the individual or all, and just spin them to max when reaching 75 C on the CPU, but they're always on. The CPU runs just fine with a sweetspot of lowest possible duty cycle 20% and 60% when 62 C and 100% when 72 C. This makes it silent even in games like CS:GO. More or less only benchs spins it up. 20% is barely noticable. Whereas the built-in "Silent" profile is more like 35-40%, and then the fan (well I always have TV/podcast/music on, but if silent it becomes like a little annoying, not as "white noise" as the bigger chassis fans).

At max the chassis' fans really makes the CPU fan not so audible. It all mixes together. Though people buying this chassis typically do serious overclocking, and say the chassis fans are a bit loud and replace them with Noctua ones (without LEDs but one can always add RGB strips or whatever if wanted, set to white always if wanted). AND/OR they primarily use water cooling which it has plenty of room for (radiator vertical or top, everything like the HDD/SSD mounts and 3 slots CD-ROM sized module can easily be removed, or one can stack the 2 x 3 HDD/SSD mounts on top of each other instead of side by side).

They're all awesome, no screws, and 2.5 disks is not a problem, just remove one snap pin and screw in the 4 SSD's screws from below. There's also room for 4 x 2.5 disks behind the back door, which are 2.5 only so snaps in, so in total 10 SATA disk slots, and 2 x M.2 disks which doesn't occupy SATA cables.... And of course in the 3 x CD-ROM module you can stack in a ton but they're not snap-ins. Haven't checked but may very well be screw holes for at least 3.5 disks. Otherwise some cheap stuff of eBay for a nice mount. Really just depends on SATA ports on MB and PSU SATA cables... UEFI supports hot-plug of SATA disks, but haven't tried. Wonder if it works...

I actually have an ASUS CD+RW ROM, not the cheapest but the cheapest ASUS black nice-looking one I found which was almost the cheapest. I really bought for my previous penguin chassis when buying Skylake (so it's pretty new and as fast as CD+RWs today) because the penguin chassis didn't come with a cover there, just a hole, or we put in one on a LAN decades ago for Linux server live ISO, maybe losing the cover, if so IDK where it is so if going to build in it again must buy a CD-ROM or maybe my cover from the Corsair fits hah.

Before it was a white one which we used a permanent marker to more or less align the colors (see CD-ROM below).

Finishing touches at my LAN crew partner's house. State of the art camera then with diskette slot for easy image transfer. Printed Bill Gates on crappy printer and cut sloppy to make it look like it is being chewed on:

Tux1

Tux2

Sorry, I won't have time for the code just yet, monster laptop packed away, but Linux installed. Won't boot since HDD in 2nd slot so must also be swapped. It's really time to put in the second PCIe4 NVMe and try RAID 0 from UEFI and install a clean installation. Also going to Spain on Tuesday and won't bring the monster laptop. Have CRON daily backups of necessary system folders using rsync and actually use Deja-Dup (GNOME 3's program called "Backups" for the home folder excluding folders not necessary like the huge hidden Gradle folder for Android Studio, the SDK, the Android VMs, and Downloads folder, never know what's there of huge files, all easy to reinstall/download). Works well. At least reliable. Kind of Time Machine for macOS, going back in time to day X. My CRON script has a folder for each day of the week so at least 7 days back. HDDs LUKSv2 encrypted backs up once in a while (manually) to LUKSv2 encrypted external USB drives.

cyring commented 4 years ago

  1. To simulate Per Core RAPL with this Xeon processor, code has been temporary modified for data injection.
    2019-10-14-200900_804x644_scrot

  2. The sum of each CPU measurement has to be equaled to the value of Cores: (in footer),
    whatever the unit (Watt, Joule) and interval (sec) are.
    This exemple: 6.1035 + 6.1035 + 12.2070 + ... + 6.1035 = 109.8630

  3. Version 1.67.3 is releasing Per Core RAPL for AMD Zen architecture.

    Tests instructions

    • Tests should fully stress all CPUs and check the Sum of CPUs against Cores
    • Other idle and mixed cases would be useful.
adatum commented 4 years ago

Per core values do seem to sum to the summary Cores value, in all-core load, single-core load, and idle conditions.

Power and Energy both report the same values. ([$] Key)

percore_power

cyring commented 4 years ago

Looks great, thank you I'm now waiting for Ryzen 3000 results to pursue with the cosmetics: masking zero values of temps and accus, single Vcore....

olejon commented 4 years ago

Ryzen 3rd Gen + X570 seems fine (or does it?) - here under stress. Under idle we're talking 0.00x Watts per Core and Total way below Package (0.xxx Watts Cores Total vs Package 13 Watts). Package Total still seem to match Ryzen Master at idle and stress and TDP. Don't think RM shows Per Core Watts, IIRC. See comment below.

Didn't do precise math, but 12.6 * 6 = 75.6, pretty below total Package but might be the expected result (?) because of hyperthreading, boost etc, other Power Usage the CPU use than just to Cores. Package Total is correct as before:

Image

NUC/Intel however is still showing 0, but still a total of > 4 Watts calculated from Cores Total it seems (like if per Cores should be really 1.xxx Watts). At both Idle and Stress it does this, but strangely again calculates Cores Total it seems anyway. Package is still correct:

Image

olejon commented 4 years ago

Sorry for all the comments, didn't see 2700X screenshot properly first. Deleted them.

Seems like screenshot from @adatum shows Cores Total SHOULD match Package Total?

As seen below, idle in this case but as seen above also under stress, and written above, Cores Total don't match Package Total on 3600X/X570. If this is a bug or simply the different architecture/platform/CPU, IDK.

NUC calculating a total still strange when Cores show 0?

Not on the NUC does Cores vs Package Total Watts match either (0.0086 Watts Cores vs 0.9 Watts Package).

Image

Image

IIRC Ryzen Master doesn't show Watts per Core, but I might be wrong. Gotta see next time I boot Windows. Probably an update to RM now as well. But can't promise when I'm gonna boot Windows the next time :P Could check out other tools that show more details, like popular HWMonitor, but I never trusted those readings for MB and CPU as their last release was before the X570 + Ryzen 3rd Gen was released, a release that was a very well hidden release from AMD, and really made Intel scared and others happy (AMD back in the game, competition!).

Causing large queues at physical stores in Taiwan and US, which doesn't happen much today in (still the few) physical shops selling computer parts directly. Maybe because they knew all the websites would already have tons of preorders. Still looked like an iPhone launch back in the days with iSheep waiting all night lol.

Maybe HWMonitor has gotten an update now. They tend to be the go-to tool for many, so maybe they even get some help from AMD for correct readings if they don't manage it properly.

cyring commented 4 years ago

@olejon : thanks a lot. To clarify the difference between Intel and AMD RAPL registers : the first is per socket counter whereas the last is per physical core (thus no SMT values). However AMD is per socket for Package. Intel specs are telling us that Package counter gathers socket and other RAM, Uncore, perhaps SOC, iGPU

The remaining programming task is to mask contextually the zero values...

NUC is an issue : the issue may come from the Joule and Watt units also probed from registers. These adjustments are undocumented and it will take several calibration tests. I believe most low power processors will be a painful journey to program.

cyring commented 4 years ago

Specification update of Mobile 4th Gen Intel Core Processor Family: See the Errata HSM82 about RAPL: is your NUC updated with the latest BIOS ?

olejon commented 4 years ago

Did update it 1-2 months back. Can check again later. Intel is very good at supporting NUCs even from 2013 with improvements and security fixes, so maybe a new one is out BUT I doubt it. They own the entire shebang of chipset, MB and iGPU so they can do a lot if they find a bug or bottleneck.

NUCs should be one of the best documented Intel complete solutions out there, all kinds of weirdness one can do with them, at least some later models than mine (I jumped on board quickly), like their answer to Raspberry Pi. Their cheapest one now isn't very expensive and also supported by Ubuntu Core (2 of them are, cheapest out, and latest powerful and most expensive, basically).

They're of course supported by Intel's own Clear Linux, which you probably should check out on an all-Intel PC, most are supported I think, at least consumer grade. That distro kind of trashes all others in benchs on Phoronix for instance, and has more and more packages, plenty for a complete desktop. But not for a daily driver. Still I wished I had tried it on my workstation. My NUC is a little busy and headless...

Just RAM and disk is your choice, and optional Wi-Fi/Bluetooth.

cyring commented 4 years ago

Did update it 1-2 months back. Can check again later. Intel is very good at supporting NUCs even from 2013 with improvements and security fixes, so maybe a new one is out BUT I doubt it. They own the entire shebang of chipset, MB and iGPU so they can do a lot if they find a bug or bottleneck.

NUCs should be one of the best documented Intel complete solutions out there, all kinds of weirdness one can do with them, at least some later models than mine (I jumped on board quickly), like their answer to Raspberry Pi. Their cheapest one now isn't very expensive and also supported by Ubuntu Core (2 of them are, cheapest out, and latest powerful and most expensive, basically).

They're of course supported by Intel's own Clear Linux, which you probably should check out on an all-Intel PC, most are supported I think, at least consumer grade. That distro kind of trashes all others in benchs on Phoronix for instance, and has more and more packages, plenty for a complete desktop. But not for a daily driver. Still I wished I had tried it on my workstation. My NUC is a little busy and headless...

Just RAM and disk is your choice, and optional Wi-Fi/Bluetooth.

It could be interesting to compare with the PAPI measurements. You may have the PAPI package in your Linux distribution or clone/build. PAPI depends on the msr.ko driver and probably Perf.

cyring commented 4 years ago

As a reminder: MSR 0x64d for the SOC Power Domain added on the development roadmap.

olejon commented 4 years ago

I'll see what I can do and check in to that when I have time, if it isn't to time expensive. Watching a documentary now (news & documentary slave here).

Just one of the cool things one can do with NUCs for instance; buy a cheap HDMI-CEC chip and connect it to the right pins on the (tiny) MB, and boom it appears as a HDMI-CEC adapter in Kodi (formerly XBMC), and such worked in harmony with my AV Receiver and TV when I used it as an HTPC. RPi has HDMI-CEC out of the box though. Last spec sheet I saw NUCs says HDMI-CEC supported (with mod). Also a USB to HDMI passthrough can be used for those less tech savvy.

Having a BIOS update in 2019, with this CPU from 2012 I think, shows that Intel cares about these things at least. And Canonical probably supported them officially from Ubuntu 12.04, if not 14.04 for sure. And then surely Red Hat. That means 2 major industry players supporting this thing for over 6 years now.

That's actually better than a Mac Mini... Which it's kind of a brilliant substitution for if you want that form factor as a desktop PC, for instance. No doubt it will be supported on Ubuntu 20.04 as well so 12+ years of official Linux support (Intel supported Linux on it from day 1, kind of why I bought it, comes with no OS, no "MS tax"). Probably best tech investment ever, considering price, different usage over the years, power consumption, just worked and works, as server or desktop. Has plenty of I/O ports... Went for the i3 model over the Celeron for future security, and SATA SSD version (which is now cheaper than the one with M.2), since there was no M.2 and the expansion disk card thing was kind of, well, bad reviews. M.2 NVMEs didn't exist basically. Fast but cheap enough SSD, didn't need 1 TB or anything, and kind of blew my mind, just like, very easy to upgrade RAM and disk, how nice, quiet and fast it was, even with "Ultra low TDP". And everything just worked, and still does. Only think lacking is UEFI support for BIOS updates through the Internet directly. Have to physically move it too, to get it to a monitor so. And cable and keyboard and USB with BIOS. Very easy and well made process, and as said still BIOS updates, just the hassle in my case now as a server.

Anyway would be really cool if you checked out Clear Linux (separate disk or whatever). CoreFreq should compile I assume with little deps. And benchs against Arch etc. AFAIK it's also a rolling release distro. Highly optimized for Intel.

cyring commented 4 years ago

Anyway would be really cool if you checked out Clear Linux (separate disk or whatever).

I would prefer you create another issue for this request but CoreFreq is dkms ready. dkms.conf released here One has to follow this Clear Linux procedure
Thanks to a nice contribution, Makefile has been enhanced with install and module_install
Just type make help

cyring commented 4 years ago

As a reminder: MSR 0x64d for the SOC Power Domain added on the development roadmap.

As mentioned In the SDM specifications, there is no guarantee this counter exists for the listed architecture families. Tested with a Skylake i7-6700, this MSR returns a zero value on all cores.

cyring commented 4 years ago
adatum commented 4 years ago

corefreq1676

cyring commented 4 years ago

@adatum : thank you

adatum commented 4 years ago

Looks like the voltages per core are independent, but the threads on the same core (threads 2 & 10 in screenshot) show the same voltage:

corefreq_single_cpu_voltage

cyring commented 4 years ago

Looks like the voltages per core are independent, but the threads on the same core (threads 2 & 10 in screenshot) show the same voltage:

My understanding of the Zen SMT architecture is that CPU 10 is the logical peer of the core CPU 2, and they have the Vcore in common. The view is showing the relation between them.

adatum commented 4 years ago

CPU 10 is the logical peer of the core CPU 2, and they have the Vcore in common.

Yes, that's what I meant to highlight. It makes sense for the two virtual cores (what I meant by "threads") to have the same Vcore since it is the same physical core. I'm not sure if that necessarily has to be the case, but at least it makes sense.