search

speed47 / spectre-meltdown-checker

Reptar, Downfall, Zenbleed, ZombieLoad, RIDL, Fallout, Foreshadow, Spectre, Meltdown vulnerability/mitigation checker for Linux & BSD
3.87k stars 476 forks source link

ibpb_enabled=2 on AMD EPYC (RHEL 7.4) #152

Closed knweiss closed 6 years ago

knweiss commented 6 years ago

Here's the spectre-meltdown-checker output of an AMD EPYC system with the latest Spectre microcode:

Spectre and Meltdown mitigation detection tool v0.35

Checking for vulnerabilities on current system
Kernel is Linux 3.10.0-693.17.1.el7.x86_64 #1 SMP Thu Jan 25 20:13:58 UTC 2018 x86_64
CPU is AMD EPYC 7451 24-Core Processor

Hardware check
* Hardware support (CPU microcode) for mitigation techniques
  * Indirect Branch Restricted Speculation (IBRS)
    * SPEC_CTRL MSR is available:  NO 
    * CPU indicates IBRS capability:  NO 
  * Indirect Branch Prediction Barrier (IBPB)
    * PRED_CMD MSR is available:  YES 
    * CPU indicates IBPB capability:  YES  (IBPB_SUPPORT feature bit)
  * Single Thread Indirect Branch Predictors (STIBP)
    * SPEC_CTRL MSR is available:  NO 
    * CPU indicates STIBP capability:  NO 
  * Enhanced IBRS (IBRS_ALL)
    * CPU indicates ARCH_CAPABILITIES MSR availability:  NO 
    * ARCH_CAPABILITIES MSR advertises IBRS_ALL capability:  NO 
  * CPU explicitly indicates not being vulnerable to Meltdown (RDCL_NO):  NO 
  * CPU microcode is known to cause stability problems:  NO 
* CPU vulnerability to the three speculative execution attacks variants
  * Vulnerable to Variant 1:  YES 
  * Vulnerable to Variant 2:  YES 
  * Vulnerable to Variant 3:  NO 

CVE-2017-5753 [bounds check bypass] aka 'Spectre Variant 1'
* Kernel has array_index_mask_nospec:  NO 
* Kernel has the Red Hat/Ubuntu patch:  YES 
> STATUS:  NOT VULNERABLE  (Kernel source has been patched to mitigate the vulnerability (Red Hat/Ubuntu patch))

CVE-2017-5715 [branch target injection] aka 'Spectre Variant 2'
* Mitigation 1
  * Kernel is compiled with IBRS/IBPB support:  YES 
  * Currently enabled features
    * IBRS enabled for Kernel space:  NO  (IBPB used instead of IBRS in all kernel entrypoints)
    * IBRS enabled for User space:  NO  (IBPB used instead of IBRS in all kernel entrypoints)
    * IBPB enabled:  YES  (IBPB used instead of IBRS in all kernel entrypoints) 
* Mitigation 2
  * Kernel compiled with retpoline option:  NO 
  * Kernel compiled with a retpoline-aware compiler:  NO 
> STATUS:  NOT VULNERABLE  (Full IBPB is mitigating the vulnerability)

CVE-2017-5754 [rogue data cache load] aka 'Meltdown' aka 'Variant 3'
* Kernel supports Page Table Isolation (PTI):  YES 
* PTI enabled and active:  NO 
* Running as a Xen PV DomU:  NO 
> STATUS:  NOT VULNERABLE  (your CPU vendor reported your CPU model as not vulnerable)

A false sense of security is worse than no security at all, see --disclaimer

What's interesting about this is the fact that Red Hat uses the following defaults on AMD CPUs:

# grep . /sys/kernel/debug/x86/*enabled
/sys/kernel/debug/x86/ibpb_enabled:2
/sys/kernel/debug/x86/ibrs_enabled:0
/sys/kernel/debug/x86/pti_enabled:0

Quote (emphasis mine): _"If ibpb_enabled is set to 2, indirect branch prediction barriers are used instead of IBRS at all kernel and hypervisor entry points (in fact, this setting also forces ibrs_enabled to 0). ibpb_enabled=2 is the default on CPUs that don’t have the SPEC_CTRL feature but only IBPB_SUPPORT. ibpb_enabled=2 doesn’t protect the kernel against attacks based on simultaneous multi-threading (SMT, also known as hyperthreading); therefore, ibpb_enabled=2 provides less complete protection unless SMT is also disabled. This feature addresses CVE-2017-5715, variant #2."_

I wonder if spectre-meltdown-checker should report this fact if SMT is enabled (it is on this test system!). Also, the "NOT VULNERABLE" rating is probably too good in this case.

knweiss commented 6 years ago

Some infos about the CPU:

# lscpu
Architecture:          x86_64
CPU op-mode(s):        32-bit, 64-bit
Byte Order:            Little Endian
CPU(s):                96
On-line CPU(s) list:   0-95
Thread(s) per core:    2
Core(s) per socket:    24
Socket(s):             2
NUMA node(s):          8
Vendor ID:             AuthenticAMD
CPU family:            23
Model:                 1
Model name:            AMD EPYC 7451 24-Core Processor
Stepping:              2
CPU MHz:               2300.000
CPU max MHz:           2300.0000
CPU min MHz:           1200.0000
BogoMIPS:              4599.47
Virtualization:        AMD-V
L1d cache:             32K
L1i cache:             64K
L2 cache:              512K
L3 cache:              8192K
NUMA node0 CPU(s):     0-5,48-53
NUMA node1 CPU(s):     6-11,54-59
NUMA node2 CPU(s):     12-17,60-65
NUMA node3 CPU(s):     18-23,66-71
NUMA node4 CPU(s):     24-29,72-77
NUMA node5 CPU(s):     30-35,78-83
NUMA node6 CPU(s):     36-41,84-89
NUMA node7 CPU(s):     42-47,90-95
Flags:                 fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx mmxext fxsr_opt pdpe1gb rdtscp lm constant_tsc art rep_good nopl nonstop_tsc extd_apicid amd_dcm aperfmperf eagerfpu pni pclmulqdq monitor ssse3 fma cx16 sse4_1 sse4_2 movbe popcnt aes xsave avx f16c rdrand lahf_lm cmp_legacy svm extapic cr8_legacy abm sse4a misalignsse 3dnowprefetch osvw skinit wdt tce topoext perfctr_core perfctr_nb bpext perfctr_l2 cpb hw_pstate ibpb_support avic fsgsbase bmi1 avx2 smep bmi2 rdseed adx smap clflushopt sha_ni xsaveopt xsavec xgetbv1 arat npt lbrv svm_lock nrip_save tsc_scale vmcb_clean flushbyasid decodeassists pause
filter pfthreshold overflow_recov succor smca
# cat /proc/cpuinfo
processor       : 0
vendor_id       : AuthenticAMD
cpu family      : 23
model           : 1
model name      : AMD EPYC 7451 24-Core Processor
stepping        : 2
microcode       : 0x8001213
cpu MHz         : 2300.000
cache size      : 512 KB
physical id     : 0
siblings        : 48
core id         : 0
cpu cores       : 24
apicid          : 0
initial apicid  : 0
fpu             : yes
fpu_exception   : yes
cpuid level     : 13
wp              : yes
flags           : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx mmxext fxsr_opt pdpe1gb rdtscp lm constant_tsc art rep_good nopl nonstop_tsc extd_apicid amd_dcm aperfmperf eagerfpu pni pclmulqdq monitor ssse3 fma cx16 sse4_1 sse4_2 movbe popcnt aes xsave avx f16c rdrand lahf_lm cmp_legacy svm extapic cr8_legacy abm sse4a misalignsse 3dnowprefetch osvw skinit wdt tce topoext perfctr_core perfctr_nb bpext perfctr_l2 cpb hw_pstate ibpb_support avic fsgsbase bmi1 avx2 smep bmi2 rdseed adx smap clflushopt sha_ni xsaveopt xsavec xgetbv1 arat npt lbrv svm_lock nrip_save tsc_scale vmcb_clean flushbyasid decodeassists pausefilter pfthreshold overflow_recov succor smca
bogomips        : 4599.47
TLB size        : 2560 4K pages
clflush size    : 64
cache_alignment : 64
address sizes   : 48 bits physical, 48 bits virtual
power management: ts ttp tm hwpstate cpb eff_freq_ro [13] [14]
[...]
speed47 commented 6 years ago

Thanks for the output dump, I implemented the AMD stuff blindly from AMD's documentation and kernel sources, but it seems it works as expected. About SMT, that's and interesting check to add. Grepping for ht from /proc/cpuinfo won't be enough as it just shows that the CPU does support HT, regardless of the fact that it's enabled or not. But checking for physical id / siblings & co should work.

For cases like that, were the protection is not completely "state of the art", but getting to the maximum protection would basically kill performance, I think I should add a --paranoid parameter, that will say "NOT VULNERABLE" only if the mitigations are enabled to the maximum extent (same stuff goes for Skylake and retpoline, which might not be enough, IBRS without IBPB, etc).

Then we would get "VULNERABLE" "PARTLY VULNERABLE" and "NOT VULNERABLE".

knweiss commented 6 years ago

Regarding HT: Yes, siblings != cpu cores in /proc/cpuinfo should do the trick.

Except for the three different states I think it would be important to have a nice, human-readable explanation, too. Many people seem to struggle with the current brief (but correct!) summaries.

What about adding URL references to official and more verbose documentation?

speed47 commented 6 years ago

I've pushed a new explain branch in an attempt to answer the question "okay, and what should I do?" in front of a VULNERABLE status on a vulnerability. Hopefully it'll help! (not yet merged to master)

mattvw commented 6 years ago

@speed47: Would this also apply to cases where you are using ibpb_enabled=1 on AMD EPYC systems to help cushion the performance impacts (this is on a RHEL 7.4 system with kernel 3.10.0-693.17.1.el7.x86_64). This is what AMD recommends and is what Red Hat says AMD recommends too (https://access.redhat.com/solutions/3335501), although it does limit the protection somewhat.

Currently, this is the output the script gives (which is VERY misleading) in this case:

[root@host ~]# grep . /sys/kernel/debug/x86/*_enabled
/sys/kernel/debug/x86/ibpb_enabled:1
/sys/kernel/debug/x86/ibrs_enabled:0
/sys/kernel/debug/x86/pti_enabled:0
[root@host ~]# ./spectre-meltdown-checker.sh 
Spectre and Meltdown mitigation detection tool v0.36

Checking for vulnerabilities on current system
Kernel is Linux 3.10.0-693.17.1.el7.x86_64 #1 SMP Sun Jan 14 10:36:03 EST 2018 x86_64
CPU is AMD EPYC 7451 24-Core Processor

Hardware check
* Hardware support (CPU microcode) for mitigation techniques
  * Indirect Branch Restricted Speculation (IBRS)
    * SPEC_CTRL MSR is available:  NO 
    * CPU indicates IBRS capability:  NO 
  * Indirect Branch Prediction Barrier (IBPB)
    * PRED_CMD MSR is available:  YES 
    * CPU indicates IBPB capability:  YES  (IBPB_SUPPORT feature bit)
  * Single Thread Indirect Branch Predictors (STIBP)
    * SPEC_CTRL MSR is available:  NO 
    * CPU indicates STIBP capability:  NO 
  * Enhanced IBRS (IBRS_ALL)
    * CPU indicates ARCH_CAPABILITIES MSR availability:  NO 
    * ARCH_CAPABILITIES MSR advertises IBRS_ALL capability:  NO 
  * CPU explicitly indicates not being vulnerable to Meltdown (RDCL_NO):  NO 
  * CPU microcode is known to cause stability problems:  NO 
* CPU vulnerability to the three speculative execution attack variants
  * Vulnerable to Variant 1:  YES 
  * Vulnerable to Variant 2:  YES 
  * Vulnerable to Variant 3:  NO 

CVE-2017-5753 [bounds check bypass] aka 'Spectre Variant 1'
* Kernel has array_index_mask_nospec:  NO 
* Kernel has the Red Hat/Ubuntu patch:  YES 
> STATUS:  NOT VULNERABLE  (Kernel source has been patched to mitigate the vulnerability (Red Hat/Ubuntu patch))

CVE-2017-5715 [branch target injection] aka 'Spectre Variant 2'
* Mitigation 1
  * Kernel is compiled with IBRS/IBPB support:  YES 
  * Currently enabled features
    * IBRS enabled for Kernel space:  NO 
    * IBRS enabled for User space:  NO 
    * IBPB enabled:  YES 
* Mitigation 2
  * Kernel compiled with retpoline option:  NO 
  * Kernel compiled with a retpoline-aware compiler:  NO 
> STATUS:  VULNERABLE  (Your kernel is compiled with IBRS but your CPU microcode is lacking support to successfully mitigate the vulnerability)

CVE-2017-5754 [rogue data cache load] aka 'Meltdown' aka 'Variant 3'
* Kernel supports Page Table Isolation (PTI):  YES 
* PTI enabled and active:  NO 
* Running as a Xen PV DomU:  NO 
> STATUS:  NOT VULNERABLE  (your CPU vendor reported your CPU model as not vulnerable)

A false sense of security is worse than no security at all, see --disclaimer

Even though IBPB is still (partially) enabled (as indicated in the * IBPB enabled: YES line) and the firmware to activate the mitigations is there (see ibpb_support CPU flag below). AMD doesn't support IBRS, so not sure why it is saying I need that support (IBPB should be sufficient, even if only partially enabled).

[root@host ~]# cat /proc/cpuinfo | grep flags | uniq
flags       : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx mmxext fxsr_opt pdpe1gb rdtscp lm constant_tsc art rep_good nopl nonstop_tsc extd_apicid amd_dcm aperfmperf eagerfpu pni pclmulqdq monitor ssse3 fma cx16 sse4_1 sse4_2 movbe popcnt aes xsave avx f16c rdrand lahf_lm cmp_legacy svm extapic cr8_legacy abm sse4a misalignsse 3dnowprefetch osvw skinit wdt tce topoext perfctr_core perfctr_nb bpext perfctr_l2 cpb hw_pstate ibpb_support avic fsgsbase bmi1 avx2 smep bmi2 rdseed adx smap clflushopt sha_ni xsaveopt xsavec xgetbv1 arat npt lbrv svm_lock nrip_save tsc_scale vmcb_clean flushbyasid decodeassists pausefilter pfthreshold overflow_recov succor smca

If I set ibpb_enabled=2 on the same system, then it shows it's fully mitigated:

[root@host ~]# echo 2 > /sys/kernel/debug/x86/ibpb_enabled
[root@host ~]# grep . /sys/kernel/debug/x86/*_enabled
/sys/kernel/debug/x86/ibpb_enabled:2
/sys/kernel/debug/x86/ibrs_enabled:0
/sys/kernel/debug/x86/pti_enabled:0
[root@host ~]# ./spectre-meltdown-checker.sh 
Spectre and Meltdown mitigation detection tool v0.36

Checking for vulnerabilities on current system
Kernel is Linux 3.10.0-693.17.1.el7.x86_64 #1 SMP Sun Jan 14 10:36:03 EST 2018 x86_64
CPU is AMD EPYC 7451 24-Core Processor

Hardware check
* Hardware support (CPU microcode) for mitigation techniques
  * Indirect Branch Restricted Speculation (IBRS)
    * SPEC_CTRL MSR is available:  NO 
    * CPU indicates IBRS capability:  NO 
  * Indirect Branch Prediction Barrier (IBPB)
    * PRED_CMD MSR is available:  YES 
    * CPU indicates IBPB capability:  YES  (IBPB_SUPPORT feature bit)
  * Single Thread Indirect Branch Predictors (STIBP)
    * SPEC_CTRL MSR is available:  NO 
    * CPU indicates STIBP capability:  NO 
  * Enhanced IBRS (IBRS_ALL)
    * CPU indicates ARCH_CAPABILITIES MSR availability:  NO 
    * ARCH_CAPABILITIES MSR advertises IBRS_ALL capability:  NO 
  * CPU explicitly indicates not being vulnerable to Meltdown (RDCL_NO):  NO 
  * CPU microcode is known to cause stability problems:  NO 
* CPU vulnerability to the three speculative execution attack variants
  * Vulnerable to Variant 1:  YES 
  * Vulnerable to Variant 2:  YES 
  * Vulnerable to Variant 3:  NO 

CVE-2017-5753 [bounds check bypass] aka 'Spectre Variant 1'
* Kernel has array_index_mask_nospec:  NO 
* Kernel has the Red Hat/Ubuntu patch:  YES 
> STATUS:  NOT VULNERABLE  (Kernel source has been patched to mitigate the vulnerability (Red Hat/Ubuntu patch))

CVE-2017-5715 [branch target injection] aka 'Spectre Variant 2'
* Mitigation 1
  * Kernel is compiled with IBRS/IBPB support:  YES 
  * Currently enabled features
    * IBRS enabled for Kernel space:  NO  (IBPB used instead of IBRS in all kernel entrypoints)
    * IBRS enabled for User space:  NO  (IBPB used instead of IBRS in all kernel entrypoints)
    * IBPB enabled:  YES  (IBPB used instead of IBRS in all kernel entrypoints)
* Mitigation 2
  * Kernel compiled with retpoline option:  NO 
  * Kernel compiled with a retpoline-aware compiler:  NO 
> STATUS:  NOT VULNERABLE  (Full IBPB is mitigating the vulnerability)

CVE-2017-5754 [rogue data cache load] aka 'Meltdown' aka 'Variant 3'
* Kernel supports Page Table Isolation (PTI):  YES 
* PTI enabled and active:  NO 
* Running as a Xen PV DomU:  NO 
> STATUS:  NOT VULNERABLE  (your CPU vendor reported your CPU model as not vulnerable)

A false sense of security is worse than no security at all, see --disclaimer

Any way this can be fixed please so that it isn't so misleading and shows some appropriate output for being partially mitigated?

Thanks!

knweiss commented 6 years ago

@mattvw First of all two comments:

  1. Please update to kernel 3.10.0-693.21.1.el7.x86_64 which has retpoline support. See #160.
  2. Please notice that the Red Hat defaults for AMD CPUs that I've mentioned in the original issue text have changed with the new kernel:
    • pti=0 ibrs=0 ibpb=1 retp=1 -> fix variant 1 and 2 if the microcode update is applied
    • pti=0 ibrs=2 ibpb=1 retp=1 -> fix variant 1 and 2 on older processors that can disable indirect branch prediction without microcode updates
mattvw commented 6 years ago

@knweiss: Yes, I am aware that there is a new kernel that introduced retpoline support and am aware of the new Red Hat defaults. I'm actually running that on quite a few other systems I manage as well. However, I also manage systems that will be sticking with the 3.10.0-693.17.1.el7.x86_64 kernel for the foreseeable future due to certain requirements/reasons (decision made by higher ups in organization), which is why I am asking about this (for me and possibly others in my situation).

We did do some testing using the new kernel with retpoline support and unfortunately weren't seeing much performance improvements using retpoline versus using IBPB mitigation, at least for our workflows.

But if @speed47 decides to not fix this since it's for an older kernel, then I understand too. But it would be nice if it could be fixed nonetheless.

speed47 commented 6 years ago

@mattvw @knweiss Can you try the branch spectre2 ? I've made a lot of changes around how the status of this vulnerability is decided, and refined the "how to fix" messages, depending on the observed CPU (Intel / AMD / ARM / other), including Skylake vs non-Skylake. It also supports the Red Hat specifics (retp_enabled, etc.). It'll also contradict by design what some kernels have to say on their own (i.e. retpoline without IBPB sometimes being considered secure enough).

@mattvw The purpose of this script is to work for all kernels and distros, old or new ones. That's actually the point: if it only gives meaningful info for latest kernels, then it would be mostly useless!

Can you try the new branch and give me some feedback ? ( https://raw.githubusercontent.com/speed47/spectre-meltdown-checker/spectre2/spectre-meltdown-checker.sh )

speed47 commented 6 years ago

@knweiss I've implemented check for SMT when ibpb_enabled=2 in the spectre2 branch. I don't have an AMD EPYC to test it against, so if you still have a system with ibpb_enabled=2 and could test it on it, that would be great ;) I also detect AMD Zen family processors (Ryzen and EPYC) to skip the IBRS explanation on them, as they won't have it (not needed), where older AMD processors should have it at some point according to AMD's documentation (already added the detection of the corresponding CPUID bit, which is not the same one as Intel's).

knweiss commented 6 years ago

@speed47 On EPYC I noticed this with code in the spectre2 branch:

./spectre-meltdown-checker_branch_spectre2.sh: line 2320: is_cpu_skylake: command not found

What about skipping this check on non-ARM?

Kernel has branch predictor hardening (arm)

Full output:

Spectre and Meltdown mitigation detection tool v0.36+                                                                                                                                                                           [3/644]

Checking for vulnerabilities on current system
Kernel is Linux 3.10.0-693.21.1.el7.x86_64 #1 SMP Wed Mar 7 19:03:37 UTC 2018 x86_64
CPU is AMD EPYC 7451 24-Core Processor

Hardware check
* Hardware support (CPU microcode) for mitigation techniques
  * Indirect Branch Restricted Speculation (IBRS)
    * SPEC_CTRL MSR is available:  NO 
    * CPU indicates IBRS capability:  NO 
    * CPU indicates preferring IBRS always-on:  NO 
    * CPU indicates preferring IBRS over retpoline:  NO 
  * Indirect Branch Prediction Barrier (IBPB)
    * PRED_CMD MSR is available:  YES 
    * CPU indicates IBPB capability:  YES  (IBPB_SUPPORT feature bit)
  * Single Thread Indirect Branch Predictors (STIBP)
    * SPEC_CTRL MSR is available:  NO 
    * CPU indicates STIBP capability:  NO 
    * CPU indicates preferring STIBP always-on:  NO 
  * CPU microcode is known to cause stability problems:  NO  (model 1 stepping 2 ucode 0x8001213 cpuid 0x800f12)
* CPU vulnerability to the three speculative execution attack variants
  * Vulnerable to Variant 1:  YES 
  * Vulnerable to Variant 2:  YES 
  * Vulnerable to Variant 3:  NO 

CVE-2017-5753 [bounds check bypass] aka 'Spectre Variant 1'
* Mitigated according to the /sys interface:  YES  (Mitigation: Load fences)
* Kernel has array_index_mask_nospec (x86):  NO 
* Kernel has the Red Hat/Ubuntu patch:  YES 
* Kernel has mask_nospec64 (arm):  NO 
> STATUS:  NOT VULNERABLE  (Mitigation: Load fences)

CVE-2017-5715 [branch target injection] aka 'Spectre Variant 2'
* Mitigated according to the /sys interface:  YES  (Mitigation: Full retpoline)
* Mitigation 1
  * Kernel is compiled with IBRS support:  YES 
    * IBRS enabled and active:  NO 
  * Kernel is compiled with IBPB support:  YES 
    * IBPB enabled and active:  YES 
* Mitigation 2
  * Kernel has branch predictor hardening (arm):  NO 
  * Kernel compiled with retpoline option:  YES 
    * Kernel compiled with a retpoline-aware compiler:  YES  (kernel reports full retpoline compilation)
    * Retpoline is enabled:  YES 
./spectre-meltdown-checker_branch_spectre2.sh: line 2320: is_cpu_skylake: command not found
> STATUS:  NOT VULNERABLE  (Full retpoline + IBPB are mitigating the vulnerability)

CVE-2017-5754 [rogue data cache load] aka 'Meltdown' aka 'Variant 3'
* Mitigated according to the /sys interface:  YES  (Not affected)
* Kernel supports Page Table Isolation (PTI):  YES 
  * PTI enabled and active:  NO 
  * Reduced performance impact of PTI:  NO  (PCID/INVPCID not supported, performance impact of PTI will be significant)
* Running as a Xen PV DomU:  NO 
> STATUS:  NOT VULNERABLE  (your CPU vendor reported your CPU model as not vulnerable)

A false sense of security is worse than no security at all, see --disclaimer

# grep . /sys/devices/system/cpu/vulnerabilities/*
/sys/devices/system/cpu/vulnerabilities/meltdown:Not affected
/sys/devices/system/cpu/vulnerabilities/spectre_v1:Mitigation: Load fences
/sys/devices/system/cpu/vulnerabilities/spectre_v2:Mitigation: Full retpoline

# grep . /sys/kernel/debug/x86/*enabled
/sys/kernel/debug/x86/ibpb_enabled:1
/sys/kernel/debug/x86/ibrs_enabled:0
/sys/kernel/debug/x86/pti_enabled:0
/sys/kernel/debug/x86/retp_enabled:1

I try to change the settings to test ibpb_enabled but it is not possible anymore:

# echo "2" >/sys/kernel/debug/x86/ibpb_enabled
-bash: echo: write error: Invalid argument
# echo "0" >/sys/kernel/debug/x86/retp_enabled                                                                                                                                                                        
# grep . /sys/kernel/debug/x86/*enabled
/sys/kernel/debug/x86/ibpb_enabled:0
/sys/kernel/debug/x86/ibrs_enabled:0
/sys/kernel/debug/x86/pti_enabled:0
/sys/kernel/debug/x86/retp_enabled:0

Notice that the 2nd echo changed ibpb_enabled to 0!

# ./spectre-meltdown-checker_branch_spectre2.sh
Spectre and Meltdown mitigation detection tool v0.36+
Checking for vulnerabilities on current system
Kernel is Linux 3.10.0-693.21.1.el7.x86_64 #1 SMP Wed Mar 7 19:03:37 UTC 2018 x86_64
CPU is AMD EPYC 7451 24-Core Processor

Hardware check
* Hardware support (CPU microcode) for mitigation techniques
  * Indirect Branch Restricted Speculation (IBRS)
    * SPEC_CTRL MSR is available:  NO 
    * CPU indicates IBRS capability:  NO 
    * CPU indicates preferring IBRS always-on:  NO 
    * CPU indicates preferring IBRS over retpoline:  NO 
  * Indirect Branch Prediction Barrier (IBPB)
    * PRED_CMD MSR is available:  YES 
    * CPU indicates IBPB capability:  YES  (IBPB_SUPPORT feature bit)
  * Single Thread Indirect Branch Predictors (STIBP)
    * SPEC_CTRL MSR is available:  NO 
    * CPU indicates STIBP capability:  NO 
    * CPU indicates preferring STIBP always-on:  NO 
  * CPU microcode is known to cause stability problems:  NO  (model 1 stepping 2 ucode 0x8001213 cpuid 0x800f12)
* CPU vulnerability to the three speculative execution attack variants
  * Vulnerable to Variant 1:  YES 
  * Vulnerable to Variant 2:  YES 
  * Vulnerable to Variant 3:  NO 

CVE-2017-5753 [bounds check bypass] aka 'Spectre Variant 1'
* Mitigated according to the /sys interface:  YES  (Mitigation: Load fences)
* Kernel has array_index_mask_nospec (x86):  NO 
* Kernel has the Red Hat/Ubuntu patch:  YES 
* Kernel has mask_nospec64 (arm):  NO 
> STATUS:  NOT VULNERABLE  (Mitigation: Load fences)

CVE-2017-5715 [branch target injection] aka 'Spectre Variant 2'
* Mitigated according to the /sys interface:  NO  (Vulnerable)
* Mitigation 1
  * Kernel is compiled with IBRS support:  YES 
    * IBRS enabled and active:  NO 
  * Kernel is compiled with IBPB support:  YES 
    * IBPB enabled and active:  NO 
* Mitigation 2
  * Kernel has branch predictor hardening (arm):  NO 
  * Kernel compiled with retpoline option:  YES 
    * Kernel compiled with a retpoline-aware compiler:  UNKNOWN 
    * Retpoline is enabled:  NO 
> STATUS:  VULNERABLE  (retpoline+IBPB is needed to mitigate the vulnerability)

> How to fix: To mitigate this vulnerability, You need a kernel compiled with retpoline + IBPB support, with retpoline requiring a retpoline-aware compiler (re-run this script with -v to know if your version of gcc is retpoline-awa
re) and IBPB requiring hardware support from your CPU microcode.

> How to fix: Both your CPU and your kernel have IBPB support, but it is currently disabled. You may enable it with `echo 1 > /sys/kernel/debug/x86/ibpb_enabled`.

> How to fix: Your kernel is compiled with retpoline, but without a retpoline-aware compiler (re-run this script with -v to know if your version of gcc is retpoline-aware).

CVE-2017-5754 [rogue data cache load] aka 'Meltdown' aka 'Variant 3'
* Mitigated according to the /sys interface:  YES  (Not affected)
* Kernel supports Page Table Isolation (PTI):  YES 
  * PTI enabled and active:  NO 
  * Reduced performance impact of PTI:  NO  (PCID/INVPCID not supported, performance impact of PTI will be significant)
* Running as a Xen PV DomU:  NO 
> STATUS:  NOT VULNERABLE  (your CPU vendor reported your CPU model as not vulnerable)

A false sense of security is worse than no security at all, see --disclaimer

The 3rd "How to fix" regarding the retpoline-aware compiler is wrong (as it is based on UNKNOWN).

Change settings back:

# echo "1" >/sys/kernel/debug/x86/retp_enabled
# grep . /sys/kernel/debug/x86/*enabled
/sys/kernel/debug/x86/ibpb_enabled:1
/sys/kernel/debug/x86/ibrs_enabled:0
/sys/kernel/debug/x86/pti_enabled:0
/sys/kernel/debug/x86/retp_enabled:1

The echo changes ibpb_enabled back to "1" again.

speed47 commented 6 years ago

./spectre-meltdown-checker_branch_spectre2.sh: line 2320: is_cpu_skylake: command not found

Thanks, regression introduced in 6e61d3c, only when retpoline & ibpb were enabled, fixed.

What about skipping this check on non-ARM?

This is not something that is checked against the running CPU, but in the kernel image itself. You can inspect an ARM kernel from an x86 machine (with --kernel and --arch-prefix), so I can't disable this kind of tests based on the CPU we're running on, as this might not be relevant.

The 3rd "How to fix" regarding the retpoline-aware compiler is wrong (as it is based on UNKNOWN).

Thanks, should be fixed too (now when we can't tell for sure that the compiler was or was not retpoline-aware, we just omit the line entirely)

Pushed on spectre2

knweiss commented 6 years ago

Full output from updated spectre2 branch:

# grep . /sys/kernel/debug/x86/*enabled
/sys/kernel/debug/x86/ibpb_enabled:1
/sys/kernel/debug/x86/ibrs_enabled:0
/sys/kernel/debug/x86/pti_enabled:0
/sys/kernel/debug/x86/retp_enabled:1

# ./spectre-meltdown-checker_branch_spectre2.sh
Spectre and Meltdown mitigation detection tool v0.36+

Checking for vulnerabilities on current system
Kernel is Linux 3.10.0-693.21.1.el7.x86_64 #1 SMP Wed Mar 7 19:03:37 UTC 2018 x86_64
CPU is AMD EPYC 7451 24-Core Processor

Hardware check
* Hardware support (CPU microcode) for mitigation techniques
  * Indirect Branch Restricted Speculation (IBRS)
    * SPEC_CTRL MSR is available:  NO 
    * CPU indicates IBRS capability:  NO 
    * CPU indicates preferring IBRS always-on:  NO 
    * CPU indicates preferring IBRS over retpoline:  NO 
  * Indirect Branch Prediction Barrier (IBPB)
    * PRED_CMD MSR is available:  YES 
    * CPU indicates IBPB capability:  YES  (IBPB_SUPPORT feature bit)
  * Single Thread Indirect Branch Predictors (STIBP)
    * SPEC_CTRL MSR is available:  NO 
    * CPU indicates STIBP capability:  NO 
    * CPU indicates preferring STIBP always-on:  NO 
  * CPU microcode is known to cause stability problems:  NO  (model 1 stepping 2 ucode 0x8001213 cpuid 0x800f12)
* CPU vulnerability to the three speculative execution attack variants
  * Vulnerable to Variant 1:  YES 
  * Vulnerable to Variant 2:  YES 
  * Vulnerable to Variant 3:  NO 

CVE-2017-5753 [bounds check bypass] aka 'Spectre Variant 1'
* Mitigated according to the /sys interface:  YES  (Mitigation: Load fences)
* Kernel has array_index_mask_nospec (x86):  NO 
* Kernel has the Red Hat/Ubuntu patch:  YES 
* Kernel has mask_nospec64 (arm):  NO 
> STATUS:  NOT VULNERABLE  (Mitigation: Load fences)

CVE-2017-5715 [branch target injection] aka 'Spectre Variant 2'
* Mitigated according to the /sys interface:  YES  (Mitigation: Full retpoline)
* Mitigation 1
  * Kernel is compiled with IBRS support:  YES 
    * IBRS enabled and active:  NO 
  * Kernel is compiled with IBPB support:  YES 
    * IBPB enabled and active:  YES 
* Mitigation 2
  * Kernel has branch predictor hardening (arm):  NO 
  * Kernel compiled with retpoline option:  YES 
    * Kernel compiled with a retpoline-aware compiler:  YES  (kernel reports full retpoline compilation)
    * Retpoline is enabled:  YES 
> STATUS:  NOT VULNERABLE  (Full retpoline + IBPB are mitigating the vulnerability)

CVE-2017-5754 [rogue data cache load] aka 'Meltdown' aka 'Variant 3'
* Mitigated according to the /sys interface:  YES  (Not affected)
* Kernel supports Page Table Isolation (PTI):  YES 
  * PTI enabled and active:  NO 
  * Reduced performance impact of PTI:  NO  (PCID/INVPCID not supported, performance impact of PTI will be significant)
* Running as a Xen PV DomU:  NO 
> STATUS:  NOT VULNERABLE  (your CPU vendor reported your CPU model as not vulnerable)

A false sense of security is worse than no security at all, see --disclaimer

Alternative settings with disabled Retpoline:

# echo "0" >/sys/kernel/debug/x86/retp_enabled

# grep . /sys/kernel/debug/x86/*enabled                                                                                                          
/sys/kernel/debug/x86/ibpb_enabled:0
/sys/kernel/debug/x86/ibrs_enabled:0
/sys/kernel/debug/x86/pti_enabled:0
/sys/kernel/debug/x86/retp_enabled:0

# ./spectre-meltdown-checker_branch_spectre2.sh
Spectre and Meltdown mitigation detection tool v0.36+

Checking for vulnerabilities on current system
Kernel is Linux 3.10.0-693.21.1.el7.x86_64 #1 SMP Wed Mar 7 19:03:37 UTC 2018 x86_64
CPU is AMD EPYC 7451 24-Core Processor

Hardware check
* Hardware support (CPU microcode) for mitigation techniques
  * Indirect Branch Restricted Speculation (IBRS)
    * SPEC_CTRL MSR is available:  NO 
    * CPU indicates IBRS capability:  NO 
    * CPU indicates preferring IBRS always-on:  NO 
    * CPU indicates preferring IBRS over retpoline:  NO 
  * Indirect Branch Prediction Barrier (IBPB)
    * PRED_CMD MSR is available:  YES 
    * CPU indicates IBPB capability:  YES  (IBPB_SUPPORT feature bit)
  * Single Thread Indirect Branch Predictors (STIBP)
    * SPEC_CTRL MSR is available:  NO 
    * CPU indicates STIBP capability:  NO 
    * CPU indicates preferring STIBP always-on:  NO 
  * CPU microcode is known to cause stability problems:  NO  (model 1 stepping 2 ucode 0x8001213 cpuid 0x800f12)
* CPU vulnerability to the three speculative execution attack variants
  * Vulnerable to Variant 1:  YES 
  * Vulnerable to Variant 2:  YES 
  * Vulnerable to Variant 3:  NO 

CVE-2017-5753 [bounds check bypass] aka 'Spectre Variant 1'
* Mitigated according to the /sys interface:  YES  (Mitigation: Load fences)
* Kernel has array_index_mask_nospec (x86):  NO 
* Kernel has the Red Hat/Ubuntu patch:  YES 
* Kernel has mask_nospec64 (arm):  NO 
> STATUS:  NOT VULNERABLE  (Mitigation: Load fences)

CVE-2017-5715 [branch target injection] aka 'Spectre Variant 2'
* Mitigated according to the /sys interface:  NO  (Vulnerable)
* Mitigation 1
  * Kernel is compiled with IBRS support:  YES 
    * IBRS enabled and active:  NO 
  * Kernel is compiled with IBPB support:  YES 
    * IBPB enabled and active:  NO 
* Mitigation 2
  * Kernel has branch predictor hardening (arm):  NO 
  * Kernel compiled with retpoline option:  YES 
    * Retpoline is enabled:  NO 
> STATUS:  VULNERABLE  (retpoline+IBPB is needed to mitigate the vulnerability)

> How to fix: To mitigate this vulnerability, You need a kernel compiled with retpoline + IBPB support, with retpoline requiring a retpoline-aware compiler (re-run this script with -v to know if your version of gcc is retpoline-awa
re) and IBPB requiring hardware support from your CPU microcode.

> How to fix: Both your CPU and your kernel have IBPB support, but it is currently disabled. You may enable it with `echo 1 > /sys/kernel/debug/x86/ibpb_enabled`.

CVE-2017-5754 [rogue data cache load] aka 'Meltdown' aka 'Variant 3'
* Mitigated according to the /sys interface:  YES  (Not affected)
* Kernel supports Page Table Isolation (PTI):  YES 
  * PTI enabled and active:  NO 
  * Reduced performance impact of PTI:  NO  (PCID/INVPCID not supported, performance impact of PTI will be significant)
* Running as a Xen PV DomU:  NO 
> STATUS:  NOT VULNERABLE  (your CPU vendor reported your CPU model as not vulnerable)

A false sense of security is worse than no security at all, see --disclaimer

You should probably change the sys path in the 2nd "Hot to fix" message from /sys/kernel/debug/x86/ibpb_enabled to /sys/kernel/debug/x86/retp_enabled (this will enable ibpb, too!) as /sys/kernel/debug/x86/ibpb_enabled is read-only:

# ls -l /sys/kernel/debug/x86/ibpb_enabled
-r-------- 1 root root 0 Apr  9 14:37 /sys/kernel/debug/x86/ibpb_enabled

Otherwise it looks good AFAICS. Thanks!

speed47 commented 6 years ago

Thanks for your feedback! I pushed a new commit to detect a read-only ibpb_enabled and tell about retp_enabled when this is the case. With this change and everything that has been piling up since the last release, I'll probably tag v0.37 tomorrow or the day after that if I get no reports of regressions in the meantime.

speed47 commented 6 years ago

Closing this, as the issue is fixed. Please reopen if needed!