Unable to adjust Turbo Frequencies on E5-1650 V2 Processor

[Gamebag]

When I try to change turbo frequencies to any value, I get this message.

However, it applies the turbo frequency to the MSR read from i7z, but it is still ignored by the system if higher than 3.6 GHz on any core.

If there is any more information you need I'd be glad to provide it.

[cyring]

Hello, Do you confirm you are following this path ?

Processor > Turbo Boost > 1C

And Turbo ratios are unlocked for this Xeon ?

[Gamebag]

That is a Yes to both questions. When I am in windows, I am able to overclock my Xeon processor with their XTU utility.

[cyring]

That is a Yes to both questions. When I am in windows, I am able to overclock my Xeon processor with their XTU utility.

OK, try this driver argument:

insmod corefreqk.ko Turbo_Ratio_Unlock=1

and post (in Markdown/code) the output of:

corefreq-cli -s

[cyring]

• TODO for future version: STH unlocked Xeon list for specific tables addition into corefreqk.h

[Gamebag]

Turbo_Ratio_Unlock is equal to 1 but no change has happened

Processor                            [Intel(R) Xeon(R) CPU E5-1650 v2 @ 3.50GHz]
|- Architecture                                                   [IvyBridge/EP]
|- Vendor ID                                                      [GenuineIntel]
|- Microcode                                                        [0x0000042e]
|- Signature                                                           [  06_3E]
|- Stepping                                                            [      4]
|- Online CPU                                                          [ 12/ 12]
|- Base Clock                                                          [ 99.759]
|- Frequency            (MHz)                      Ratio                        
                 Min   1197.11                    <  12 >                       
                 Max   3491.58                    <  35 >                       
|- Factory                                                             [100.000]
                       3500                       [  35 ]                       
|- Performance                                                                  
   |- P-State                                                                   
                 TGT   3990.37                    <  40 >                       
|- Turbo Boost                                                         [ UNLOCK]
                  1C   3990.37                    <  40 >                       
                  2C   3990.37                    <  40 >                       
                  3C   3990.37                    <  40 >                       
                  4C   3990.37                    <  40 >                       
                  5C   3990.37                    <  40 >                       
                  6C   3990.37                    <  40 >                       
                  7C   3990.37                    <  40 >                       
                  8C   3990.37                    <  40 >                       
|- Uncore                                                              [   LOCK]
                 Min   1197.11                    [  12 ]                       
                 Max   3491.58                    [  35 ]                       
|- TDP                                                           Level [  0:0  ]
   |- Programmable                                                     [ UNLOCK]
   |- Configuration                                                    [   LOCK]
   |- Turbo Activation                                                 [   LOCK]

Instruction Set Extensions                                                      
|- 3DNow!/Ext [N/N]          ADX [N]          AES [Y]  AVX/AVX2 [Y/N] 
|- AVX512-F     [N]    AVX512-DQ [N]  AVX512-IFMA [N]   AVX512-PF [N] 
|- AVX512-ER    [N]    AVX512-CD [N]    AVX512-BW [N]   AVX512-VL [N] 
|- AVX512-VBMI  [N] AVX512-VBMI2 [N]  AVX512-VNMI [N]  AVX512-ALG [N] 
|- AVX512-VPOP  [N] AVX512-VNNIW [N] AVX512-FMAPS [N] AVX512-VP2I [N] 
|- AVX512-BF16  [N]  BMI1/BMI2 [N/N]         CLWB [N] CLFLUSH/O [Y/N] 
|- CLAC-STAC    [N]         CMOV [Y]    CMPXCHG8B [Y]  CMPXCHG16B [Y] 
|- F16C         [Y]          FPU [Y]         FXSR [Y]   LAHF-SAHF [Y] 
|- MMX/Ext    [Y/N] MON/MWAITX [Y/N]        MOVBE [N]   PCLMULQDQ [Y] 
|- POPCNT       [Y]       RDRAND [Y]       RDSEED [N]      RDTSCP [Y] 
|- SEP          [Y]          SHA [N]          SSE [Y]        SSE2 [Y] 
|- SSE3         [Y]        SSSE3 [Y]  SSE4.1/4A [Y/N]      SSE4.2 [Y] 
|- SERIALIZE    [N]      SYSCALL [Y]          SGX [N]       RDPID [N] 

Features                                                                        
|- 1 GB Pages Support                                      1GB-PAGES   [Capable]
|- Advanced Configuration & Power Interface                     ACPI   [Capable]
|- Advanced Programmable Interrupt Controller                   APIC   [Capable]
|- Core Multi-Processing                                  CMP Legacy   [Missing]
|- L1 Data Cache Context ID                                  CNXT-ID   [Missing]
|- Direct Cache Access                                           DCA   [Capable]
|- Debugging Extension                                            DE   [Capable]
|- Debug Store & Precise Event Based Sampling               DS, PEBS   [Capable]
|- CPL Qualified Debug Store                                  DS-CPL   [Capable]
|- 64-Bit Debug Store                                         DTES64   [Capable]
|- Fast-String Operation                                Fast-Strings   [Capable]
|- Fused Multiply Add                                     FMA | FMA4   [Missing]
|- Hardware Lock Elision                                         HLE   [Missing]
|- Instruction Based Sampling                                    IBS   [Missing]
|- Long Mode 64 bits                                       IA64 | LM   [Capable]
|- LightWeight Profiling                                         LWP   [Missing]
|- Machine-Check Architecture                                    MCA   [Capable]
|- Memory Protection Extensions                                  MPX   [Missing]
|- Model Specific Registers                                      MSR   [Capable]
|- Memory Type Range Registers                                  MTRR   [Capable]
|- OS-Enabled Ext. State Management                          OSXSAVE   [Capable]
|- Physical Address Extension                                    PAE   [Capable]
|- Page Attribute Table                                          PAT   [Capable]
|- Pending Break Enable                                          PBE   [Capable]
|- Process Context Identifiers                                  PCID   [Capable]
|- Perfmon and Debug Capability                                 PDCM   [Capable]
|- Page Global Enable                                            PGE   [Capable]
|- Page Size Extension                                           PSE   [Capable]
|- 36-bit Page Size Extension                                  PSE36   [Capable]
|- Processor Serial Number                                       PSN   [Missing]
|- Resource Director Technology/PQE                            RDT-A   [Missing]
|- Resource Director Technology/PQM                            RDT-M   [Missing]
|- Restricted Transactional Memory                               RTM   [Missing]
|- Safer Mode Extensions                                         SMX   [Capable]
|- Self-Snoop                                                     SS   [Capable]
|- Supervisor-Mode Access Prevention                            SMAP   [Missing]
|- Supervisor-Mode Execution Prevention                         SMEP   [Capable]
|- Time Stamp Counter                                            TSC [Invariant]
|- Time Stamp Counter Deadline                          TSC-DEADLINE   [Capable]
|- TSX Force Abort MSR Register                            TSX-ABORT   [Missing]
|- TSX Suspend Load Address Tracking                       TSX-LDTRK   [Missing]
|- User-Mode Instruction Prevention                             UMIP   [Missing]
|- Virtual Mode Extension                                        VME   [Capable]
|- Virtual Machine Extensions                                    VMX   [Capable]
|- Extended xAPIC Support                                     x2APIC   [  xAPIC]
|- Execution Disable Bit Support                              XD-Bit   [Capable]
|- XSAVE/XSTOR States                                          XSAVE   [Capable]
|- xTPR Update Control                                          xTPR   [Capable]
Mitigation mechanisms                                                           
|- Indirect Branch Restricted Speculation                       IBRS   [Capable]
|- Indirect Branch Prediction Barrier                           IBPB   [Capable]
|- Single Thread Indirect Branch Predictor                     STIBP   [Capable]
|- Speculative Store Bypass Disable                             SSBD   [Capable]
|- Writeback & invalidate the L1 data cache                L1D-FLUSH   [Capable]
|- Hypervisor - No flush L1D on VM entry            L1DFL_VMENTRY_NO   [Missing]
|- Architectural - Buffer Overwriting                       MD-CLEAR   [Capable]
|- Architectural - Rogue Data Cache Load                     RDCL_NO   [Missing]
|- Architectural - Enhanced IBRS                            IBRS_ALL   [Missing]
|- Architectural - Return Stack Buffer Alternate                RSBA   [Missing]
|- Architectural - Speculative Store Bypass                   SSB_NO   [Missing]
|- Architectural - Microarchitectural Data Sampling           MDS_NO   [Missing]
|- Architectural - TSX Asynchronous Abort                     TAA_NO   [Missing]
|- Architectural - Page Size Change MCE               PSCHANGE_MC_NO   [Missing]
|- Architectural - Split Locked Access Exception                SPLA   [Missing]

Technologies                                                                    
|- System Management Mode                                   SMM-Dual       [OFF]
|- Hyper-Threading                                               HTT       [ ON]
|- SpeedStep                                                    EIST       < ON>
|- Dynamic Acceleration                                          IDA       [ ON]
|- Turbo Boost                                                 TURBO       < ON>
|- Virtualization                                                VMX       [OFF]
   |- I/O MMU                                                   VT-d       [OFF]
   |- Hypervisor                                                           [OFF]

Performance Monitoring                                                          
|- Version                                                        PM       [  3]
|- Counters:          General                   Fixed                           
|                     4 x 48 bits             3 x 48 bits                       
|- Enhanced Halt State                                           C1E       <OFF>
|- C1 Auto Demotion                                              C1A       <OFF>
|- C3 Auto Demotion                                              C3A       <OFF>
|- C1 UnDemotion                                                 C1U       <OFF>
|- C3 UnDemotion                                                 C3U       <OFF>
|- Legacy Frequency ID control                                   FID       [OFF]
|- Legacy Voltage ID control                                     VID       [OFF]
|- Hardware-Controlled Performance States                        HWP       [OFF]
|- P-State Hardware Coordination Feedback                MPERF/APERF       [ ON]
|- Hardware Duty Cycling                                         HDC       [OFF]
|- Package C-States                                                             
   |- Configuration Control                                   CONFIG   [   LOCK]
   |- Lowest C-State                                           LIMIT   <      6>
   |- I/O MWAIT Redirection                                  IOMWAIT   < Enable>
   |- Max C-State Inclusion                                    RANGE   <      3>
|- MONITOR/MWAIT                                                                
   |- State index:    #0    #1    #2    #3    #4    #5    #6    #7              
   |- Sub C-State:     0     2     1     1     0     0     0     0              
|- Core Cycles                                                         [Capable]
|- Instructions Retired                                                [Capable]
|- Reference Cycles                                                    [Capable]
|- Last Level Cache References                                         [Capable]
|- Last Level Cache Misses                                             [Capable]
|- Branch Instructions Retired                                         [Capable]
|- Branch Mispredicts Retired                                          [Capable]

Power & Thermal                                                                 
|- Clock Modulation                                             ODCM   <Disable>
   |- DutyCycle                                                        [  0.00%]
|- Power Management                                         PWR MGMT   [   LOCK]
   |- Energy Policy                                        Bias Hint   <      4>
   |- Energy Policy                                          HWP EPP   [      0]
|- Junction Temperature                                        TjMax   [  0: 90]
|- Digital Thermal Sensor                                        DTS   [Capable]
|- Power Limit Notification                                      PLN   [Capable]
|- Package Thermal Management                                    PTM   [Capable]
|- Thermal Monitor 1                                             TM1   [ Enable]
|- Thermal Monitor 2                                             TM2   [Capable]
|- Thermal Design Power                                          TDP   [    130]
   |- Minimum Power                                              Min   [     60]
   |- Maximum Power                                              Max   [Missing]
|- Units                                                                        
   |- Power                                               watt   [  0.125000000]
   |- Energy                                             joule   [  0.000015259]
   |- Window                                            second   [  0.000976562]

The multipliers are all at 40x from previous testing. I've also realized that the CPU doesn't turbo above 3.6 even in single threads when it should be at 3.9. I don't know if its because I changed the multiplier or not.

[cyring]

• What my Xeon W3690 seems to share with your Xeon E5-1650 v2 is that they have unpublished Turbo unlock capability. W3690 is already registered as an exception into CoreFreq tables. E5-1650 v2 is something to add in source code; reason I have asked to add the argument Turbo_Ratio_Unlock=1 Check with dmesg for no spelling error warned by the driver.

• Depending on the architecture, Boost is triggered, depending on the Target P-State TGT, and off course the Turbo Boost bit enabled With Nehalem/Westmere, Target ratio must be the Max ratio + one With Sandy Bridge and above, Target ratio is any value above the Max ratio (yours is 35)

• Thus I'm finding more confortable to have a full control over the P-States by disabling any other drivers Here is my boot command line:

  nmi_watchdog=0 modprobe.blacklist=pcspkr,iTCO_wdt,acpi_cpufreq,pcc_cpufreq,intel_cstate,intel_uncore,intel_powerclamp,i7core_edac,i5500_temp,coretemp,asus_atk0110 idle=halt intel_pstate=disable cpufreq.off=0

• Then I load CoreFreq driver with those arguments:

  insmod corefreqk.ko RDPMC_Enable=1 Register_ClockSource=1 Register_Governor=1 Register_CPU_Freq=1 Register_CPU_Idle=1 Turbo_Ratio_Unlock=1

  Note: Turbo_Ratio_Unlock=1 is added for your case

• Next just start the Daemon and the Client with the following usage in the UI

  1. Turbo is OFF
  2. Enable Turbo
  3. Turbo is ON You are noticing that the Target ratio TGT has been increased
  4. Change the single Core Turbo ratio in 1C Select the first line Processor because ratio is common to all Cores Here going to 4 GHz The new frequency is confirmed in both windows at 1C
  5. Stress a single Core Press O for Tools and go to Turbo Select CPU... Enter one of your Core. Here CPU number 0 Press ESC to exit any window and observe the resulted frequency Observe that ruler has been updated with the new frequency ratio Press F10 to stop the Stress

• Turbo Boost Prerequisites:

  1. Turbo bit unlocked (or overrided is supported)
  2. SpeedStep ON
  3. C-States ON
  4. IDA capable
  5. Turbo ON
  6. P-State ratio well defined
  7. Clock Modulation ODCM is disabled

[Gamebag]

I followed your steps but when I started the program I noticed turbo was still enabled. I then followed the rest of your steps ignoring that and I was still met with "turbo must be disabled". However, I am now getting the turbo boost to 3.9GHz I am supposed to on single core.

[Gamebag]

I tried again and while I was able to get it to start without turbo and then being able to enable turbo, changing the frequencies still results in the turbo must be disabled message

[Gamebag]

Turning up the TGT to 40 makes the system respect the default turbo settings

[cyring]

• I believe I have a remaining issue synchronizing the Turbo state in the UI from changes happening in Driver. At any time, you can press key * to request a full refresh

• I had programmed those atomic algorithms in Tools to provide enough stress to trigger Turbo. It may explain why it now reaches 4 GHz

• For multi Cores or high Temperature, I recommend one of my Conics stress loops

• If not done yet, disable SPREAD SPECTRUM in your BIOS to get a better 100 MHz Base Clock in CoreFreq

• Tomorrow, you should get a CoreFreq update which cares about your Xeon

[Gamebag]

My bad I meant that it now respects the 39/37/37/36/36/36 default Xeon settings. I am not getting requested 40x turbo frequencies. SPREAD SPECTRUM is not a thing in my BIOS, as I am using an HP Z420 Workstation. I wonder whats causing it to ignore my settings in Linux but do just fine in Windows.

[cyring]

Hello,

Can you switch to the develop branch

You will get some specific codes to unlock your Processor; thus you don't need any driver argument.

As a reminder, don't forget to fully rebuild and reload all (Driver, Daemon, Cli) when new code is available.

make clean all
insmod corefreqk.ko
corefreqd
corefreq-cli

[Gamebag]

I have switched to develop branch, but I am still getting the same "Turbo Boost must be disabled" message. However, uncore is now unlocked

[cyring]

Can you please pull the latest develop and try again ?

[Gamebag]

You've fixed the Turbo must be enabled message! However the CPU is still only boosting to 3.6GHz.

[cyring]

You've fixed the Turbo must be enabled message! However the CPU is still only boosting to 3.6GHz.

So Processor is able to reach its 3.9 GHz nominal Turbo boost frequency but can not go above ?

[Gamebag]

I don't know what changed but single core only boosts to 3.6 again unfortunately

[cyring]

Start the single Core stress tools on CPU 0 and let it run. It the mean time, toogle the features below, one after one:

TGT = Highest ratio 1C = Your requested ratio 2C, 3C, ... 7C, 8C = Default boot ratios BIAS Hint = 0 (no power savings) ODCM = Disable IOMWAIT = Disable

---

2nd use case (still stressing) Turbo OFF EIST OFF EIST ON Turbo ON

[Gamebag]

IOMWAIT does not disable on my machine in this software. I followed your 2nd use case and the frequency went back up from base right to 3.6 again

[cyring]

Let's try the Semaphore bit. New code available in develop for your tests

[Gamebag]

Still boosting to 3.6 max

[cyring]

It's the only relevant topic I have found on the subject and the SDM specs say:

Semaphore for Turbo Ratio Limit Configuration If 1, the processor uses override configuration1 specified in MSR_TURBO_RATIO_LIMIT and MSR_TURBO_RATIO_LIMIT1. If 0, the processor uses factory-set configuration (Default).

[Gamebag]

At this point this feels like a linux thing. You guys have tried everything and it just refuses to boost above 3.6

[Gamebag]

I still have all the CPU drivers blacklisted for what its worth

[cyring]

My bad. Please pull develop and try again.

EDIT: if despite everything latest code does not work; here they are saying that change has to done before loading microcode !

[Gamebag]

IT WORKS! Thank you so much! You're a genius!

[cyring]

Thank you for providing me all these tests. Main things I have learned from EP processors:

1. purpose of the Semaphore bit
2. EP with less than 8 physical Cores requires conditional Turbo limits MSR writing
3. Some EP Xeons are unofficially Turbo unlocked

Later I'll will code back to you with code refactoring and clean up.

Meanwhile feel free to try any other CoreFreq features, including screenshots.

State of:

• Memory controller
• All expected Package C-States are showing up
• Clock Modulation with percent of your choice is limiting frequency
• Accuracy of Temps, Voltages, Energy counters
• Temperature of the whole Processor (in footer) vs value per Core
• Effectiveness of the CPU-Idle to ask Cores for deep sleeping
• Limiting the C-States level in the Kernel window
• Any value compared with BIOS or OS, such as Microcode version, etc

Enhancements:

• IOMWAIT, I probably have to white-list your architecture
• Find/Add the Voltage register of the whole Processor vs Voltage Core
• White-list Processor in Power Mgmt for BIAS hint changed with the Frequency/Sensors effects

Qualifying and tuning CoreFreq for a x86 architecture may takes us several days. I hope you're OK to pursue this work.

btw a star is welcomed (if not done yet)

[Gamebag]

I can confirm some of them now. All package C-States are showing up, Clock modulation works, Temp monitoring works very well, CPU-Idle also works well on this, All the BIOS / OS values that I know the value of are right, Voltage monitoring is correct. I haven't tried limiting C-states yet.

I have starred the program :+1: It's very good!

[cyring]

Thanks a lot for all your tests. Do you mind to show me a screenshot of the Memory Controller ? (those registers are indeed tuff to find)

[Gamebag]

Sure!

[cyring]

Thanks

I have to work on:

• DRAM speed
• Missing timings on last 2 channels

So, you have been able to overclock Turbo to 4.2 GHz ;-)

[Gamebag]

Yes! Turbo overclocking works perfectly now! Once voltage is implemented it'll be unstoppable

[cyring]

Yes! Turbo overclocking works perfectly now! Once voltage is implemented it'll be unstoppable

I've been looking in datasheet a way to alter the VID but unfortunately Voltages appears to be read-only. Same with the Nuveton SuperIO controller.

[Gamebag]

In XTU there's a slider for additional voltage in Turbo mode. I wonder if this is undocumented as well?

[cyring]

In XTU there's a slider for additional voltage in Turbo mode. I wonder if this is undocumented as well?

Read IA32_PERF_CTL msr register at 0x199 each time the voltage is slide up/down. Reproduce the same write by hand to verify if change is acknowledged.

Most upper bits of IA32_PERF_CTL are left reserved in https://software.intel.com/content/www/us/en/develop/download/intel-64-and-ia-32-architectures-sdm-combined-volumes-1-2a-2b-2c-2d-3a-3b-3c-3d-and-4.html

Here's the state of my research so far: https://github.com/cyring/CoreFreq/blob/5da83ae6c6691eea5ba22834e4fb0d7392cce18c/intelmsr.h#L422

[cyring]

Hello,

EDIT: based on latest develop where refresh has been fixed, is the DRAM speed still missing ? Then can you post the output of:

lspci -nn

Also, how many DIMMs are there ? How the slots are populated (including the empty ones) ?

Thank you.

[Gamebag]

Speed is still missing. There are 4 DIMMS, in positions 1,3,5,7. Unpopulated are 2,4,6,8. my LSPCI is here: lspci.txt

[cyring]

Hello

Can you pull and test the latest develop branch which should fix the bus rate speed.

About DRAM speed, it may be an unspecified value and I will need your help to add a print into code at this line: https://github.com/cyring/CoreFreq/blob/eef2d33ca0143effc7c3d348f33f97547618eae2/corefreqd.c#L3322 insert above that line:

printf("DMFC=%x\n", Proc->Uncore.Bus.SNB_EP_Cap1.DMFC);

Value will be printed after building, and starting Daemon.

About frequency, if you can't get it from BIOS, try dmidecode -t memory

Thank you

(SmartPhone edition)

[cyring]

Maybe one day, please help with above question.

[islampro400]

activating turbo boost on z420 can you help me

[cyring]

activating turbo boost on z420 can you help me

Ok, first what is z420 ? Can you tell/show where you end up with CoreFreq: Build? Run?

If running, please provides a minimum of corefreq-cli -s