search

open-power / op-build

Buildroot overlay for Open Power
GNU General Public License v2.0
103 stars 183 forks source link

STB: P9 IPL doesn't mention key transition start and finish messages on to the console. #1890

Open pridhiviraj opened 6 years ago

pridhiviraj commented 6 years ago

In P8 time frame Host firmware used to notify the user about Key Transition start and finish when we initiate a IPL after flashing key transition PNOR.

   61.21569|sbe|Performing Secureboot Key Transition
   61.21569|IPMI: shutdown complete
   61.21570|sbe|System will power off after completion
   61.45656|Stopping istep dispatcher

In P9 systems, i am able to do the key transition, but in console no way user knows when the transition started and finished. Below are the IPL messages for P9 system. 

--== Welcome to Hostboot hostboot-de81205/hbicore.bin ==--

  2.78852|secure|SecureROM valid - enabling functionality
 14.75399|Ignoring boot flags, incorrect version 0x0
 14.76097|Booting from SBE side 0 on master proc=00050000
 14.83979|ISTEP  6. 5 - host_init_fsi
 14.94902|ISTEP  6. 6 - host_set_ipl_parms
 15.02514|ISTEP  6. 7 - host_discover_targets
 23.67760|HWAS|PRESENT> DIMM[03]=F0F0000000000000
 23.67761|HWAS|PRESENT> Proc[05]=8800000000000000
 23.67762|HWAS|PRESENT> Core[07]=FC3CCFCF0F3F0000
 23.69700|ISTEP  6. 8 - host_update_master_tpm
 31.24082|SECURE|Security Access Bit> 0xC000000000000000
 31.24082|SECURE|Secure Mode Disable (via Jumper)> 0x0000000000000000
 31.24110|ISTEP  6. 9 - host_gard
 31.26474|HWAS|FUNCTIONAL> DIMM[03]=F0F0000000000000
 31.26475|HWAS|FUNCTIONAL> Proc[05]=8800000000000000
 31.26475|HWAS|FUNCTIONAL> Core[07]=FC3CCFCF0F3F0000
 31.27024|ISTEP  6.10 - host_revert_sbe_mcs_setup
 31.28226|ISTEP  6.11 - host_start_occ_xstop_handler
 32.00369|ISTEP  6.12 - host_voltage_config
 32.06074|ISTEP  7. 1 - mss_attr_cleanup
 32.21375|ISTEP  7. 2 - mss_volt
 32.29042|ISTEP  7. 3 - mss_freq
 32.45020|ISTEP  7. 4 - mss_eff_config
 34.02501|ISTEP  7. 5 - mss_attr_update
 34.03703|ISTEP  8. 1 - host_slave_sbe_config
 34.09321|ISTEP  8. 2 - host_setup_sbe
 34.09896|ISTEP  8. 3 - host_cbs_start
 34.12158|ISTEP  8. 4 - proc_check_slave_sbe_seeprom_complete
 38.33917|ISTEP  8. 5 - host_attnlisten_proc
 38.35342|ISTEP  8. 6 - host_p9_fbc_eff_config
 38.35934|ISTEP  8. 7 - host_p9_eff_config_links
 38.37012|ISTEP  8. 8 - proc_attr_update
 38.37445|ISTEP  8. 9 - proc_chiplet_fabric_scominit
 38.40630|ISTEP  8.10 - proc_xbus_scominit
 40.21341|ISTEP  8.11 - proc_xbus_enable_ridi
 40.22168|ISTEP  8.12 - host_set_voltages
 40.26091|ISTEP  9. 1 - fabric_erepair
 40.32220|ISTEP  9. 2 - fabric_io_dccal
 41.01888|ISTEP  9. 3 - fabric_pre_trainadv
 41.02530|ISTEP  9. 4 - fabric_io_run_training
 41.15234|ISTEP  9. 5 - fabric_post_trainadv
 41.15785|ISTEP  9. 6 - proc_smp_link_layer
 41.16671|ISTEP  9. 7 - proc_fab_iovalid
 41.21944|ISTEP  9. 8 - host_fbc_eff_config_aggregate
 41.23180|ISTEP 10. 1 - proc_build_smp
 41.35477|ISTEP 10. 2 - host_slave_sbe_update
 42.62879|sbe|System Performing SBE Update for PROC 0, side 0
 66.37281|sbe|System Performing SBE Update for PROC 1, side 0

 89.77820|Stopping istep dispatcher

  1. System is in imprint mode:

    [root@ltc-boston125 ~]# ipmitool fru print 47 Product Name : OpenPOWER Firmware Product Version : open-power-SUPERMICRO-P9DSU-V1.03-20180205-imp Product Extra : op-build-a05d69b-dirty Product Extra : skiboot-v5.9-240-g081882690163-pcbedce4 Product Extra : hostboot-9bfb201 Product Extra : linux-4.14.13-openpower1-p78d7eee Product Extra : petitboot-v1.6.6-p019c87e Product Extra : machine-xml-fb5f933 Product Extra : occ-

  2. After Key transition to production

    / # ipmitool fru print 47 Product Name : OpenPOWER Firmware Product Version : open-power-p9dsu-v1.21-rc2-dirty-prod Product Extra : buildroot-2017.11-5-g65679be Product Extra : skiboot-v5.10-rc3 Product Extra : hostboot-de81205 Product Extra : linux-4.14.16-openpower1-p0d02e12 Product Extra : petitboot-v1.6.6-pf2406aa Product Extra : machine-xml-fb5f933 Product Extra : occ-f72f857 Product Extra : hostboot-binarie / #

dcrowell77 commented 6 years ago

Something for @bofferdn

bofferdn commented 6 years ago

I doubt this actually was a key transition driver; the procedure to make a key transition driver that installs development keys is:

op-build BR2_OPENPOWER_SECUREBOOT_NO_KEY_TRANSITION=n BR2_OPENPOWER_SECUREBOOT_KEY_TRANSITION_TO_PROD=n BR2_OPENPOWER_SECUREBOOT_KEY_TRANSITION_TO_DEV=y openpower-pnor-rebuild

(assuming you have already have a development driver sitting there built in the normal way)

And, it will show very explicit console messages indicating a key transition is occurring.

bofferdn commented 6 years ago

For what it's worth, the machine is claiming to be in secure mode.

pridhiviraj commented 6 years ago

@bofferdn As discussed in slack, i properly built the key transition driver that installs production keys. And also regarding secure mode, OPAL shows secure mode is on(Not FORCED by NVRAM).

[   72.004918096,5] STB: Found ibm,secureboot-v2
[   72.006398337,5] STB: secure mode on
[   72.008533338,5] STB: trusted mode off
bofferdn commented 6 years ago

Agree, so something weird is going on. We have had some less than ideal behavior in this istep even as soon as today, but the known fixes are in master stream, so would have to investigate whether those are present or not in this driver. Otherwise, we verified the compile settings, so it does seem odd. I'd try a few times in dev->dev mode and see if if the results change at all, and then we can talk a bit more / see what else we need to do.

pridhiviraj commented 6 years ago

@bofferdn I tried flashing dev-to-dev key transition PNOR to do the key recovery process, But still i see same IPL message and got shutdown after SBE update, i didn't see any key transition start and finish messages on console. But the key transition went fine and system properly transition from production to imprint mode..

--== Welcome to Hostboot hostboot-28927a7/hbicore.bin ==--

  2.71990|secure|SecureROM valid - enabling functionality
 13.82401|secure|Booting in non-secure mode.
 14.54192|Ignoring boot flags, incorrect version 0x0
 14.54750|Booting from SBE side 0 on master proc=00050000
 14.64966|ISTEP  6. 5 - host_init_fsi
 14.73520|ISTEP  6. 6 - host_set_ipl_parms
 14.80827|ISTEP  6. 7 - host_discover_targets
 23.50322|HWAS|PRESENT> DIMM[03]=F0F0000000000000
 23.50323|HWAS|PRESENT> Proc[05]=8800000000000000
 23.50324|HWAS|PRESENT> Core[07]=FC3CCFCF0F3F0000
 23.52084|ISTEP  6. 8 - host_update_master_tpm
 30.85903|SECURE|Security Access Bit> 0x0000000000000000
 30.85904|SECURE|Secure Mode Disable (via Jumper)> 0xC000000000000000
 30.85947|ISTEP  6. 9 - host_gard
 30.88488|HWAS|FUNCTIONAL> DIMM[03]=F0F0000000000000
 30.88489|HWAS|FUNCTIONAL> Proc[05]=8800000000000000
 30.88490|HWAS|FUNCTIONAL> Core[07]=FC3CCFCF0F3F0000
 30.88819|ISTEP  6.10 - host_revert_sbe_mcs_setup
 30.90015|ISTEP  6.11 - host_start_occ_xstop_handler
 31.54980|ISTEP  6.12 - host_voltage_config
 31.62940|ISTEP  7. 1 - mss_attr_cleanup
 31.87621|ISTEP  7. 3 - mss_freq
 33.04599|ISTEP  7. 4 - mss_eff_config
 34.50302|ISTEP  7. 5 - mss_attr_update
 34.51343|ISTEP  8. 1 - host_slave_sbe_config
 34.55438|ISTEP  8. 2 - host_setup_sbe
 34.55977|ISTEP  8. 3 - host_cbs_start
 34.57881|ISTEP  8. 4 - proc_check_slave_sbe_seeprom_complete
 38.79637|ISTEP  8. 5 - host_attnlisten_proc
 38.80127|ISTEP  8. 6 - host_p9_fbc_eff_config
 38.80762|ISTEP  8. 7 - host_p9_eff_config_links
 38.81637|ISTEP  8. 8 - proc_attr_update
 38.82470|ISTEP  8. 9 - proc_chiplet_fabric_scominit
 38.85176|ISTEP  8.10 - proc_xbus_scominit
 40.49715|ISTEP  8.11 - proc_xbus_enable_ridi
 40.50381|ISTEP  8.12 - host_set_voltages
 40.54000|ISTEP  9. 1 - fabric_erepair
 40.58290|ISTEP  9. 2 - fabric_io_dccal
 41.38161|ISTEP  9. 5 - fabric_post_trainadv
 41.38677|ISTEP  9. 6 - proc_smp_link_layer
 41.39540|ISTEP  9. 7 - proc_fab_iovalid
 41.41838|ISTEP  9. 8 - host_fbc_eff_config_aggregate
 41.43262|ISTEP 10. 1 - proc_build_smp
 41.55193|ISTEP 10. 2 - host_slave_sbe_update
 42.12957|sbe|System Performing SBE Update for PROC 0, side 0
 65.51166|sbe|System Performing SBE Update for PROC 1, side 0
ete

 88.82367|Stopping istep dispatcher
hellerda commented 6 years ago

So, it appears now that Pridhiviraj has confirmed that all the key transition PNORs are in fact working, in secure mode or out of secure mode, and it's just the message that are missing. So hopefully it won't be hard to determine why it was working previously (for P8 builds) but is not working now.

pridhiviraj commented 6 years ago

With latest upstream op-build PNOR build on witherspoon it works fine.

cat /var/lib/phosphor-software-manager/pnor/ro/VERSION 
open-power-witherspoon-v1.22-rc1-1-gc61ecab
    buildroot-2017.11.2-8-g4b6188e
    skiboot-v5.11-rc1
    hostboot-6eaa457
    linux-4.15.9-openpower1-p497d1fe
    petitboot-v1.7.1-pa873880
    machine-xml-c10638f-p35bfee7
    occ-768466b
    hostboot-binaries-2657e58
    capp-ucode-p9-dd2-v3
    sbe-5c03639

Console messages:


--== Welcome to Hostboot hostboot-6eaa457/hbicore.bin ==--

  4.48311|secure|SecureROM valid - enabling functionality
  4.49230|secure|Booting in non-secure mode.
  6.92244|Booting from SBE side 0 on master proc=00050000
  7.01228|ISTEP  6. 5 - host_init_fsi
  7.26076|ISTEP  6. 6 - host_set_ipl_parms
  7.29684|ISTEP  6. 7 - host_discover_targets
 12.49958|HWAS|PRESENT> DIMM[03]=AAAA000000000000
 12.49959|HWAS|PRESENT> Proc[05]=8800000000000000
 12.49960|HWAS|PRESENT> Core[07]=CC3F3FFFF0CC0000
 12.55716|ISTEP  6. 8 - host_update_master_tpm
 21.21787|SECURE|Security Access Bit> 0x0000000000000000
 21.21788|SECURE|Secure Mode Disable (via Jumper)> 0xC000000000000000
 21.21799|ISTEP  6. 9 - host_gard
 21.24171|HWAS|FUNCTIONAL> DIMM[03]=AAAA000000000000
 21.24173|HWAS|FUNCTIONAL> Proc[05]=8800000000000000
 21.24174|HWAS|FUNCTIONAL> Core[07]=CC3F3FFFF0CC0000
 21.24656|ISTEP  6.10 - host_revert_sbe_mcs_setup
 21.24766|ISTEP  6.11 - host_start_occ_xstop_handler
 22.54800|ISTEP  6.12 - host_voltage_config
 22.68098|ISTEP  7. 1 - mss_attr_cleanup
 23.43633|ISTEP  7. 2 - mss_volt
 23.55138|ISTEP  7. 3 - mss_freq
 25.86411|ISTEP  7. 4 - mss_eff_config
 27.03312|ISTEP  7. 5 - mss_attr_update
 27.04328|ISTEP  8. 1 - host_slave_sbe_config
 27.42083|ISTEP  8. 2 - host_setup_sbe
 27.43214|ISTEP  8. 3 - host_cbs_start
 27.46131|ISTEP  8. 4 - proc_check_slave_sbe_seeprom_complete
 34.08397|ISTEP  8. 5 - host_attnlisten_proc
 34.08494|ISTEP  8. 6 - host_p9_fbc_eff_config
 34.09062|ISTEP  8. 7 - host_p9_eff_config_links
 34.10026|ISTEP  8. 8 - proc_attr_update
 34.10195|ISTEP  8. 9 - proc_chiplet_fabric_scominit
 34.13380|ISTEP  8.10 - proc_xbus_scominit
 35.17003|ISTEP  8.11 - proc_xbus_enable_ridi
 35.17542|ISTEP  8.12 - host_set_voltages
 35.28555|ISTEP  9. 1 - fabric_erepair
 35.33792|ISTEP  9. 2 - fabric_io_dccal
 36.04955|ISTEP  9. 3 - fabric_pre_trainadv
 36.05373|ISTEP  9. 4 - fabric_io_run_training
 36.18929|ISTEP  9. 5 - fabric_post_trainadv
 36.19223|ISTEP  9. 6 - proc_smp_link_layer
 36.19928|ISTEP  9. 7 - proc_fab_iovalid
 36.42851|ISTEP  9. 8 - host_fbc_eff_config_aggregate
 36.43640|ISTEP 10. 1 - proc_build_smp
 36.57582|ISTEP 10. 2 - host_slave_sbe_update
 37.53841|sbe|System Performing SBE Update for PROC 0, side 0
 62.56895|sbe|System Performing SBE Update for PROC 1, side 0
 87.41851|sbe|Performing Secure Boot key transition

 87.41852|sbe|System will power off after completion

 87.43836|IPMI: shutdown complete

 87.53345|Stopping istep dispatcher

--== Welcome to Hostboot hostboot-6eaa457/hbicore.bin ==--

  4.48535|secure|SecureROM valid - enabling functionality
  4.49431|secure|Booting in non-secure mode.
  5.96699|Booting from SBE side 0 on master proc=00050000
  6.01615|ISTEP  6. 5 - host_init_fsi
  6.21774|ISTEP  6. 6 - host_set_ipl_parms
  6.27481|ISTEP  6. 7 - host_discover_targets
  6.77491|HWAS|PRESENT> DIMM[03]=AAAA000000000000
  6.77492|HWAS|PRESENT> Proc[05]=8800000000000000
  6.77493|HWAS|PRESENT> Core[07]=CC3F3FFFF0CC0000
  6.80632|ISTEP  6. 8 - host_update_master_tpm
 16.12482|SECURE|Security Access Bit> 0x0000000000000000
 16.12483|SECURE|Secure Mode Disable (via Jumper)> 0xC000000000000000
 16.12495|ISTEP  6. 9 - host_gard
 16.16155|HWAS|FUNCTIONAL> DIMM[03]=AAAA000000000000
 16.16156|HWAS|FUNCTIONAL> Proc[05]=8800000000000000
 16.16157|HWAS|FUNCTIONAL> Core[07]=CC3F3FFFF0CC0000
 16.16689|ISTEP  6.10 - host_revert_sbe_mcs_setup
 16.16978|ISTEP  6.11 - host_start_occ_xstop_handler
 17.06194|ISTEP  6.12 - host_voltage_config
 17.19942|ISTEP  7. 1 - mss_attr_cleanup
 17.78286|ISTEP  7. 2 - mss_volt
 17.92956|ISTEP  7. 3 - mss_freq
 20.24157|ISTEP  7. 4 - mss_eff_config
 21.42203|ISTEP  7. 5 - mss_attr_update
 21.43244|ISTEP  8. 1 - host_slave_sbe_config
 21.51987|ISTEP  8. 2 - host_setup_sbe
 21.52535|ISTEP  8. 3 - host_cbs_start
 21.55302|ISTEP  8. 4 - proc_check_slave_sbe_seeprom_complete
 28.17495|ISTEP  8. 5 - host_attnlisten_proc
 28.17598|ISTEP  8. 6 - host_p9_fbc_eff_config
 28.18169|ISTEP  8. 7 - host_p9_eff_config_links
 28.19256|ISTEP  8. 8 - proc_attr_update
 28.19462|ISTEP  8. 9 - proc_chiplet_fabric_scominit
 28.22592|ISTEP  8.10 - proc_xbus_scominit
 29.24853|ISTEP  8.11 - proc_xbus_enable_ridi
 29.25414|ISTEP  8.12 - host_set_voltages
 29.33499|ISTEP  9. 1 - fabric_erepair
 29.38954|ISTEP  9. 2 - fabric_io_dccal
 30.10260|ISTEP  9. 3 - fabric_pre_trainadv
 30.10670|ISTEP  9. 4 - fabric_io_run_training
 30.24242|ISTEP  9. 5 - fabric_post_trainadv
 30.24513|ISTEP  9. 6 - proc_smp_link_layer
 30.25251|ISTEP  9. 7 - proc_fab_iovalid
 30.48208|ISTEP  9. 8 - host_fbc_eff_config_aggregate
 30.48986|ISTEP 10. 1 - proc_build_smp
 30.62718|ISTEP 10. 2 - host_slave_sbe_update
 31.38492|sbe|System Performing SBE Update for PROC 0, side 1
 56.34201|sbe|System Performing SBE Update for PROC 1, side 1
 81.07290|sbe|Performing Secure Boot key transition

 81.07291|sbe|System will power off after completion

 81.08325|IPMI: shutdown complete

 81.14425|Stopping istep dispatcher