search

open-power / skiboot

OPAL boot and runtime firmware for POWER
Apache License 2.0
98 stars 134 forks source link

[251972.070211564,3] STB: BOOTKERNEL NOT MEASURED. Already exited from boot services #150

Open pridhiviraj opened 6 years ago

pridhiviraj commented 6 years ago

In fast-reboot path STB not measuring BOOTKERNEL partition as trusted boot checks for boot_services_exited which is set to True during first full IPL. So only verification is happening at the moment not measurement.

/ # reboot
The system is going down NOW!
Sent SIGTERM to all processes
Sent SIGKILL to all processes
Requesting system reboot
[  552.761067] reboot: Restarting system
[251964.220126908,5] OPAL: Reboot request...
[251964.221748414,5] RESET: Initiating fast reboot 4...
[251964.455192120,5] PCI: Clearing all devices...
[251964.455428149,7] LPC-MBOX: Sending BMC interrupt
[251964.456844280,7] blocklevel_read: 0x0   0x31c03b10  0x30
[251964.456846992,7] blocklevel_raw_read: 0x0   0x31c03b10  0x30
[251964.456860485,7] FFS: Partition map size: 0x1000
[251964.456861640,7] blocklevel_read: 0x0   0x30b6c438  0x1000
[251964.456863050,7] blocklevel_raw_read: 0x0   0x30b6c438  0x1000
[251964.458394442,7] FLASH: BOOTKERNEL partition doesn't have ECC
[251964.458395834,7] blocklevel_read: 0x17e1000 0x20000000  0x1000
[251964.458397354,7] blocklevel_raw_read: 0x17e1000 0x20000000  0x1000
[251964.459218438,7] FLASH: BOOTKERNEL partition is signed
[251964.459219661,7] blocklevel_read: 0x17e2000 0x20001000  0x1117448
[251964.459221233,7] blocklevel_raw_read: 0x17e2000 0x20001000  0x1117448
[251972.070211564,3] STB: BOOTKERNEL NOT MEASURED. Already exited from boot services
[251990.138816014,5] PCI: Resetting PHBs and training links...
[251992.357863493,5] PCI: Probing slots...
[251992.415675609,5] PHB#0000:00:00.0 [ROOT] 1014 04c1 R:00 C:060400 B:01..01 SLOT=UIO Slot1 
[251992.419391428,5] PHB#0000:01:00.0 [EP  ] 15b3 1019 R:00 C:020700 (       network) LOC_CODE=UIO Slot1
[251992.424356529,5] PHB#0000:01:00.1 [EP  ] 15b3 1019 R:00 C:020700 (       network) LOC_CODE=UIO Slot1
[251992.428608378,5] PHB#0001:00:00.0 [ROOT] 1014 04c1 R:00 C:060400 B:01..09 SLOT=UIO Slot2 
[251992.432867579,5] PHB#0001:01:00.0 [SWUP] 10b5 8725 R:ca C:060400 B:02..09 LOC_CODE=UIO Slot2
[251992.437133785,5] PHB#0001:02:01.0 [SWDN] 10b5 8725 R:ca C:060400 B:03..03 SLOT=S000103 
[251992.441387635,5] PHB#0001:03:00.0 [EP  ] 1000 00c9 R:01 C:010700 (           sas) LOC_CODE=S000103
[251992.446352230,5] PHB#0001:02:08.0 [SWDN] 10b5 8725 R:ca C:060400 B:04..08 
[251992.449906921,5] PHB#0001:02:09.0 [SWDN] 10b5 8725 R:ca C:060400 B:09..09 SLOT=S000109 
[251992.454164082,5] PHB#0001:09:00.0 [EP  ] 8086 1589 R:02 C:020000 (      ethernet) LOC_CODE=S000109
[251992.458425288,5] PHB#0001:09:00.1 [EP  ] 8086 1589 R:02 C:020000 (      ethernet) LOC_CODE=S000109
[251992.463393516,5] PHB#0001:09:00.2 [EP  ] 8086 1589 R:02 C:020000 (      ethernet) LOC_CODE=S000109
[251992.467657659,5] PHB#0001:09:00.3 [EP  ] 8086 1589 R:02 C:020000 (      ethernet) LOC_CODE=S000109
[251992.472620931,5] PHB#0001:01:00.1 [EP  ] 10b5 87d0 R:ca C:088000 (system-peripheral) LOC_CODE=UIO Slot2
[251992.476887046,5] PHB#0001:01:00.2 [EP  ] 10b5 87d0 R:ca C:088000 (system-peripheral) LOC_CODE=UIO Slot2
[251992.481854680,5] PHB#0001:01:00.3 [EP  ] 10b5 87d0 R:ca C:088000 (system-peripheral) LOC_CODE=UIO Slot2
[251992.486823842,5] PHB#0001:01:00.4 [EP  ] 10b5 87d0 R:ca C:088000 (system-peripheral) LOC_CODE=UIO Slot2
[251992.491794377,5] PHB#0002:00:00.0 [ROOT] 1014 04c1 R:00 C:060400 B:01..01 SLOT=Onboard LAN 
[251992.496054136,5] PHB#0002:01:00.0 [EP  ] 1000 00c9 R:01 C:010700 (           sas) LOC_CODE=Onboard LAN
[251992.501022310,5] PHB#0003:00:00.0 [ROOT] 1014 04c1 R:00 C:060400 B:01..01 SLOT=Onboard SAS 
[251992.505283317,5] PHB#0003:01:00.0 [EP  ] 9005 028d R:01 C:010700 (           sas) LOC_CODE=Onboard SAS
[251992.509547984,5] PHB#0004:00:00.0 [ROOT] 1014 04c1 R:00 C:060400 B:01..02 SLOT=Onboard BMC 
[251993.001808164,5] PHB#0004:01:00.0 [ETOX] 1a03 1150 R:04 C:060400 B:02..02 LOC_CODE=Onboard BMC
[251993.006777928,5] PHB#0004:02:00.0 [PCID] 1a03 2000 R:41 C:030000 (           vga) LOC_CODE=Onboard BMC
[251993.011038869,5] PHB#0005:00:00.0 [ROOT] 1014 04c1 R:00 C:060400 B:01..01 SLOT=Onboard USB 
[251993.016001328,5] PHB#0005:01:00.0 [EP  ] 104c 8241 R:02 C:0c0330 (      usb-xhci) LOC_CODE=Onboard USB
[251993.020268117,5] PHB#0030:00:00.0 [ROOT] 1014 04c1 R:00 C:060400 B:01..01 SLOT=WIO Slot1 
[251993.024542109,5] PHB#0030:01:00.0 [EP  ] 15b3 1019 R:00 C:020700 (       network) LOC_CODE=WIO Slot1
[251993.029507087,5] PHB#0030:01:00.1 [EP  ] 15b3 1019 R:00 C:020700 (       network) LOC_CODE=WIO Slot1
[251993.033758813,5] PHB#0031:00:00.0 [ROOT] 1014 04c1 R:00 C:060400 B:01..01 SLOT=WIO-R Slot 
[251993.038017231,5] PHB#0031:01:00.0 [EP  ] 1000 00c9 R:01 C:010700 (           sas) LOC_CODE=WIO-R Slot
[251993.042986226,5] PHB#0032:00:00.0 [ROOT] 1014 04c1 R:00 C:060400 B:01..0d SLOT=WIO Slot3 
[251993.047246365,5] PHB#0032:01:00.0 [SWUP] 10b5 8725 R:ca C:060400 B:02..0d LOC_CODE=WIO Slot3
[251993.051512715,5] PHB#0032:02:01.0 [SWDN] 10b5 8725 R:ca C:060400 B:03..03 SLOT=S003203 
[251993.055765916,5] PHB#0032:03:00.0 [EP  ] 1000 00c9 R:01 C:010700 (           sas) LOC_CODE=S003203
[251993.060729252,5] PHB#0032:02:08.0 [SWDN] 10b5 8725 R:ca C:060400 B:04..08 
[251993.064283476,5] PHB#0032:02:09.0 [SWDN] 10b5 8725 R:ca C:060400 B:09..0d SLOT=S003209 
[251993.067837232,5] PHB#0032:01:00.1 [EP  ] 10b5 87d0 R:ca C:088000 (system-peripheral) LOC_CODE=WIO Slot3
[251993.072805227,5] PHB#0032:01:00.2 [EP  ] 10b5 87d0 R:ca C:088000 (system-peripheral) LOC_CODE=WIO Slot3
[251993.077773974,5] PHB#0032:01:00.3 [EP  ] 10b5 87d0 R:ca C:088000 (system-peripheral) LOC_CODE=WIO Slot3
[251993.082741477,5] PHB#0032:01:00.4 [EP  ] 10b5 87d0 R:ca C:088000 (system-peripheral) LOC_CODE=WIO Slot3
[251993.087711802,5] PHB#0033:00:00.0 [ROOT] 1014 04c1 R:00 C:060400 B:01..01 SLOT=WIO Slot2 
[251993.091975626,5] PHB#0033:01:00.0 [EP  ] 11f8 f117 R:06 C:010802 (  mass-storage) LOC_CODE=WIO Slot2
[251993.096246779,5] IPMI: Resetting boot count on successful boot
[251993.099073528,5] INIT: Waiting for kernel...
[251993.101908360,5] INIT: 64-bit LE kernel discovered
[251993.104041014,3] STB: EV_SEPARATOR (pcr0) NOT MEASURED. No TPM registered/enabled
[251993.107592487,3] STB: EV_SEPARATOR (pcr1) NOT MEASURED. No TPM registered/enabled
[251993.111849349,3] STB: EV_SEPARATOR (pcr2) NOT MEASURED. No TPM registered/enabled
[251993.115401785,3] STB: EV_SEPARATOR (pcr3) NOT MEASURED. No TPM registered/enabled
[251993.119658493,3] STB: EV_SEPARATOR (pcr4) NOT MEASURED. No TPM registered/enabled
[251993.123211984,3] STB: EV_SEPARATOR (pcr5) NOT MEASURED. No TPM registered/enabled
[251993.127467308,3] STB: EV_SEPARATOR (pcr6) NOT MEASURED. No TPM registered/enabled
[251993.131020089,3] STB: EV_SEPARATOR (pcr7) NOT MEASURED. No TPM registered/enabled
[251993.164583851,5] INIT: Starting kernel at 0x20011000, fdt at 0x30a96c08 360328 bytes)
[251994.282717509,5] OPAL: Switch to little-endian OS
[    0.000000] opal: OPAL detected !
pridhiviraj commented 6 years ago 
/ # cat /sys/firmware/opal/msglog | grep -i STB
[   71.254280957,6] MEM: parsing reserved memory from node /ibm,hostboot/reserved-memory
[   75.394363524,3] STB: container NOT VERIFIED, resource_id=4 secureboot not yet initialized
[   76.000269556,5] STB: Found ibm,secureboot-v2
[   76.001880905,5] STB: secure mode off
[   76.004009987,6] STB: Found CVC @ 200ffd230000-200ffd23ffff
[   76.004012317,6] STB: Found CVC-sha512 @ 200ffd230040, version=1
[   76.004014213,6] STB: Found CVC-verify @ 200ffd230050, version=1
[   76.004019031,5] STB: trusted mode on
[   76.005463742,5] STB: Found tpm0,i2c_tpm_nuvoton evLogLen=2174 evLogSize=65536
[   76.150282035,5] STB: IMA_CATALOG verified
[   76.151990586,5] STB: IMA_CATALOG hash calculated
[   76.197565954,5] STB: IMA_CATALOG measured on pcr2 (tpm0, evType 0x5, evLogLen 2257)
[   76.347314490,5] STB: CAPP verified
[   76.349226963,5] STB: CAPP hash calculated
[   76.394079794,5] STB: CAPP measured on pcr2 (tpm0, evType 0x5, evLogLen 2333)
[   84.112839822,5] STB: BOOTKERNEL verified
[   84.175060791,5] STB: BOOTKERNEL hash calculated
[   84.219955974,5] STB: BOOTKERNEL measured on pcr4 (tpm0, evType 0x5, evLogLen 2415)
[   85.063476810,5] STB: EV_SEPARATOR measured on pcr0 (tpm0, evType 0x4, evLogLen 2491)
[   85.110205265,5] STB: EV_SEPARATOR measured on pcr1 (tpm0, evType 0x4, evLogLen 2567)
[   85.156551954,5] STB: EV_SEPARATOR measured on pcr2 (tpm0, evType 0x4, evLogLen 2643)
[   85.203131811,5] STB: EV_SEPARATOR measured on pcr3 (tpm0, evType 0x4, evLogLen 2719)
[   85.249990931,5] STB: EV_SEPARATOR measured on pcr4 (tpm0, evType 0x4, evLogLen 2795)
[   85.296376140,5] STB: EV_SEPARATOR measured on pcr5 (tpm0, evType 0x4, evLogLen 2871)
[   85.343028021,5] STB: EV_SEPARATOR measured on pcr6 (tpm0, evType 0x4, evLogLen 2947)
[   85.389250243,5] STB: EV_SEPARATOR measured on pcr7 (tpm0, evType 0x4, evLogLen 3023)
[ 1069.077794447,5] STB: BOOTKERNEL verified
[ 1069.079405326,3] STB: BOOTKERNEL NOT MEASURED. Already exited from boot services
[ 1069.087727317,3] STB: EV_SEPARATOR (pcr0) NOT MEASURED. No TPM registered/enabled
[ 1069.091277930,3] STB: EV_SEPARATOR (pcr1) NOT MEASURED. No TPM registered/enabled
[ 1069.094830682,3] STB: EV_SEPARATOR (pcr2) NOT MEASURED. No TPM registered/enabled
[ 1069.099085602,3] STB: EV_SEPARATOR (pcr3) NOT MEASURED. No TPM registered/enabled
[ 1069.102637912,3] STB: EV_SEPARATOR (pcr4) NOT MEASURED. No TPM registered/enabled
[ 1069.106893031,3] STB: EV_SEPARATOR (pcr5) NOT MEASURED. No TPM registered/enabled
[ 1069.110445897,3] STB: EV_SEPARATOR (pcr6) NOT MEASURED. No TPM registered/enabled
[ 1069.113998405,3] STB: EV_SEPARATOR (pcr7) NOT MEASURED. No TPM registered/enabled
/ #

From the above messages now it is clear that, in first full IPL OPAL verified and measured proeprly, but on next fast-reboot OPAL just verified it but skipped measuring it.

pridhiviraj commented 6 years ago 
diff --git a/core/fast-reboot.c b/core/fast-reboot.c
index 0fe16cc..86408f6 100644
--- a/core/fast-reboot.c
+++ b/core/fast-reboot.c
@@ -30,6 +30,8 @@
 #include <ipmi.h>
 #include <direct-controls.h>
 #include <nvram.h>
+#include "libstb/trustedboot.h"
+#include "libstb/tpm_chip.h"

 /* Flag tested by the OPAL entry code */
 static volatile bool fast_boot_release;
@@ -330,6 +332,13 @@ void __noreturn fast_reboot_entry(void)
        cpu_set_sreset_enable(true);
        cpu_set_ipi_enable(true);

+       /* We are loading the BOOTKERNEL from PNOR, in order to function
+         * trusted_measure, do tpm_init and enable boot services flag
+         */
+       boot_services_exited = false;
+
+       tpm_init();
+
        /* Start preloading kernel and ramdisk */
        start_preload_kernel();

diff --git a/libstb/drivers/tpm_i2c_nuvoton.c b/libstb/drivers/tpm_i2c_nuvoton.c
index d18add9..ed50e07 100644
--- a/libstb/drivers/tpm_i2c_nuvoton.c
+++ b/libstb/drivers/tpm_i2c_nuvoton.c
@@ -534,7 +534,7 @@ void tpm_i2c_nuvoton_probe(void)
 {
        struct tpm_dev *tpm_device = NULL;
        struct dt_node *node = NULL;
-       struct i2c_bus *bus;
+       struct i2c_bus *bus = NULL;

        dt_for_each_compatible(dt_root, node, "nuvoton,npct650") {
                if (!dt_node_is_enabled(node))
@@ -578,7 +578,7 @@ void tpm_i2c_nuvoton_probe(void)
                        continue;
                }
                bus = i2c_find_bus_by_id(tpm_device->bus_id);
-               assert(bus->check_quirk == NULL);
+               /* assert(bus->check_quirk == NULL); */
                bus->check_quirk = nuvoton_tpm_quirk;
                bus->check_quirk_data = tpm_device;

diff --git a/libstb/tpm_chip.c b/libstb/tpm_chip.c
index 2858caf..58e5f75 100644
--- a/libstb/tpm_chip.c
+++ b/libstb/tpm_chip.c
@@ -313,6 +313,7 @@ int tpm_extendl(TPM_Pcr pcr,
 void tpm_add_status_property(void) {
        struct tpm_chip *tpm;
        list_for_each(&tpm_list, tpm, link) {
+               dt_check_del_prop(tpm->node, "status");
                dt_add_property_string(tpm->node, "status",
                                       tpm->enabled ? "okay" : "disabled");
        }
diff --git a/libstb/trustedboot.c b/libstb/trustedboot.c
index 151e4e1..79e39a2 100644
--- a/libstb/trustedboot.c
+++ b/libstb/trustedboot.c
@@ -31,7 +31,7 @@

 static bool trusted_mode = false;
 static bool trusted_init = false;
-static bool boot_services_exited = false;
+bool boot_services_exited;

 /*
  * This maps a PCR for each resource we can measure. The PCR number is
diff --git a/libstb/trustedboot.h b/libstb/trustedboot.h
index 3003c80..bb4fcb6 100644
--- a/libstb/trustedboot.h
+++ b/libstb/trustedboot.h
@@ -19,6 +19,8 @@

 #include <platform.h>

+extern bool boot_services_exited;
+
 void trustedboot_init(void);

 /**

With the above changes trusted_measure is functional in fast-reboot path. But i have commented the assert check for i2c bus quirk. it's giving me an assert in fast-reboot path.


[  149.169253837,5] STB: Found tpm0,i2c_tpm_nuvoton evLogLen=3023 evLogSize=65536
[  149.172504406,0] Assert fail: libstb/drivers/tpm_i2c_nuvoton.c:581:bus->check_quirk == ((void *)0)
[  149.176768185,0] Aborting!
CPU 0818 Backtrace:
 S: 0000000033c63b90 R: 000000003001367c   .backtrace+0x48
 S: 0000000033c63c20 R: 000000003001a314   ._abort+0x4c
 S: 0000000033c63ca0 R: 000000003001a390   .assert_fail+0x34
 S: 0000000033c63d20 R: 00000000300a5534   .tpm_i2c_nuvoton_probe+0x1f8
 S: 0000000033c63df0 R: 00000000300a35c0   .tpm_init+0x3c
 S: 0000000033c63e70 R: 0000000030025a30   .fast_reboot_entry+0x2d4
 S: 0000000033c63f00 R: 0000000030002a2c   fast_reset_entry+0x2c
[  149.203762670,4] IPMI: Dropped eSEL: BMC code is buggy/missing

If i comment that line, it works fine. @stewart-ibm Why the bus->check_quirk function pointer is getting non-NULL in fast-reboot path.

pridhiviraj commented 6 years ago

After discussing with @stewart-ibm in internal slack we decided not to do tpm_init again, instead we are not un-registering the tpm chips. Send the fix to the mailing list https://lists.ozlabs.org/pipermail/skiboot/2018-March/010730.html

/ # 
/ # cat /sys/firmware/opal/msglog | grep -i STB
[   56.155311958,6] MEM: parsing reserved memory from node /ibm,hostboot/reserved-memory
[   60.267505083,3] STB: container NOT VERIFIED, resource_id=4 secureboot not yet initialized
[   60.376867311,5] STB: Found ibm,secureboot-v2
[   60.378607730,5] STB: secure mode off
[   60.380738237,6] STB: Found CVC @ 200ffd230000-200ffd23ffff
[   60.380740108,6] STB: Found CVC-sha512 @ 200ffd230040, version=1
[   60.380741958,6] STB: Found CVC-verify @ 200ffd230050, version=1
[   60.380745879,5] STB: trusted mode on
[   60.382188970,5] STB: Found tpm0,i2c_tpm_nuvoton evLogLen=2174 evLogSize=65536
[   61.009958962,5] STB: IMA_CATALOG verified
[   61.011798937,5] STB: IMA_CATALOG hash calculated
[   61.056680325,5] STB: IMA_CATALOG measured on pcr2 (tpm0, evType 0x5, evLogLen 2257)
[   61.203533454,5] STB: CAPP verified
[   61.204873792,5] STB: CAPP hash calculated
[   61.249634075,5] STB: CAPP measured on pcr2 (tpm0, evType 0x5, evLogLen 2333)
[   68.071123345,5] STB: BOOTKERNEL verified
[   68.132748898,5] STB: BOOTKERNEL hash calculated
[   68.177780040,5] STB: BOOTKERNEL measured on pcr4 (tpm0, evType 0x5, evLogLen 2415)
[   69.020288589,5] STB: EV_SEPARATOR measured on pcr0 (tpm0, evType 0x4, evLogLen 2491)
[   69.067131504,5] STB: EV_SEPARATOR measured on pcr1 (tpm0, evType 0x4, evLogLen 2567)
[   69.113500453,5] STB: EV_SEPARATOR measured on pcr2 (tpm0, evType 0x4, evLogLen 2643)
[   69.160151478,5] STB: EV_SEPARATOR measured on pcr3 (tpm0, evType 0x4, evLogLen 2719)
[   69.206314215,5] STB: EV_SEPARATOR measured on pcr4 (tpm0, evType 0x4, evLogLen 2795)
[   69.252779324,5] STB: EV_SEPARATOR measured on pcr5 (tpm0, evType 0x4, evLogLen 2871)
[   69.299546002,5] STB: EV_SEPARATOR measured on pcr6 (tpm0, evType 0x4, evLogLen 2947)
[   69.345791575,5] STB: EV_SEPARATOR measured on pcr7 (tpm0, evType 0x4, evLogLen 3023)
[  157.107870446,5] STB: BOOTKERNEL verified
[  157.170144276,5] STB: BOOTKERNEL hash calculated
[  157.220727126,5] STB: BOOTKERNEL measured on pcr4 (tpm0, evType 0x5, evLogLen 3105)
[  157.271683286,5] STB: EV_SEPARATOR measured on pcr0 (tpm0, evType 0x4, evLogLen 3181)
[  157.317878789,5] STB: EV_SEPARATOR measured on pcr1 (tpm0, evType 0x4, evLogLen 3257)
[  157.364339706,5] STB: EV_SEPARATOR measured on pcr2 (tpm0, evType 0x4, evLogLen 3333)
[  157.411026897,5] STB: EV_SEPARATOR measured on pcr3 (tpm0, evType 0x4, evLogLen 3409)
[  157.457316137,5] STB: EV_SEPARATOR measured on pcr4 (tpm0, evType 0x4, evLogLen 3485)
[  157.503873377,5] STB: EV_SEPARATOR measured on pcr5 (tpm0, evType 0x4, evLogLen 3561)
[  158.038687091,5] STB: EV_SEPARATOR measured on pcr6 (tpm0, evType 0x4, evLogLen 3637)
[  158.085073868,5] STB: EV_SEPARATOR measured on pcr7 (tpm0, evType 0x4, evLogLen 3713)

With this patch, now it measures the BOOTKERNEL partition in fast-reboot path as well.

pridhiviraj commented 6 years ago

After having a working V2 in place, but this really needs testing with fast-reboot torture testcase.

pridhiviraj commented 6 years ago

Sent a working V2 https://lists.ozlabs.org/pipermail/skiboot/2018-March/010738.html and survived 100 reboots.