Host upgrade status => kube node reboot condition

SpComb commented 6 years ago

Change the host upgrades to report status information.

This includes a RebootRequired flag, used to update the kube node HostUpgradesReboot condition.

Change the HostUpgrades condition to report:

Unknown status on UpgradeFailed with the error message
False status on RebootRequired with the upgrade log
True status on succesful upgrade + no reboot required with the upgrade log
TODO allow checking for pending upgrades and setting the condition to False with UpgradesPending

SpComb commented 6 years ago

Test with HostUpgrades output message:

$ kubectl get nodes/localhost.localdomain -o yaml
apiVersion: v1
kind: Node
...
status:
  ...
  conditions:
  - lastHeartbeatTime: 2018-05-23T16:11:04Z
    lastTransitionTime: 2018-05-23T16:11:04Z
    message: |
      exiting because "Download Only" specified
      The following updates will be downloaded on localhost.localdomain:
      ================================================================================
       Package                 Arch      Version                     Repository  Size
      ================================================================================
      Installing:
       kernel                  x86_64    3.10.0-862.3.2.el7          updates     46 M
      Updating:
       NetworkManager          x86_64    1:1.10.2-14.el7_5           updates    1.7 M
       NetworkManager-libnm    x86_64    1:1.10.2-14.el7_5           updates    1.3 M
       NetworkManager-team     x86_64    1:1.10.2-14.el7_5           updates    161 k
       NetworkManager-tui      x86_64    1:1.10.2-14.el7_5           updates    235 k
       ca-certificates         noarch    2018.2.22-70.0.el7_5        updates    392 k
       centos-release          x86_64    7-5.1804.el7.centos.2       updates     24 k
       dhclient                x86_64    12:4.2.5-68.el7.centos.1    updates    284 k
       dhcp-common             x86_64    12:4.2.5-68.el7.centos.1    updates    175 k
       dhcp-libs               x86_64    12:4.2.5-68.el7.centos.1    updates    131 k
       e2fsprogs               x86_64    1.42.9-12.el7_5             updates    699 k
       e2fsprogs-libs          x86_64    1.42.9-12.el7_5             updates    167 k
       iptables                x86_64    1.4.21-24.1.el7_5           updates    432 k
       kernel-tools            x86_64    3.10.0-862.3.2.el7          updates    6.2 M
       kernel-tools-libs       x86_64    3.10.0-862.3.2.el7          updates    6.2 M
       libcom_err              x86_64    1.42.9-12.el7_5             updates     41 k
       libgcc                  x86_64    4.8.5-28.el7_5.1            updates    101 k
       libgomp                 x86_64    4.8.5-28.el7_5.1            updates    156 k
       libss                   x86_64    1.42.9-12.el7_5             updates     45 k
       libstdc++               x86_64    4.8.5-28.el7_5.1            updates    303 k
       nspr                    x86_64    4.19.0-1.el7_5              updates    127 k
       nss                     x86_64    3.36.0-5.el7_5              updates    835 k
       nss-softokn             x86_64    3.36.0-5.el7_5              updates    315 k
       nss-softokn-freebl      x86_64    3.36.0-5.el7_5              updates    222 k
       nss-sysinit             x86_64    3.36.0-5.el7_5              updates     62 k
       nss-tools               x86_64    3.36.0-5.el7_5              updates    514 k
       nss-util                x86_64    3.36.0-1.el7_5              updates     78 k
       openldap                x86_64    2.4.44-15.el7_5             updates    355 k
       python-perf             x86_64    3.10.0-862.3.2.el7          updates    6.2 M
       rsyslog                 x86_64    8.24.0-16.el7_5.4           updates    607 k

      Transaction Summary
      ================================================================================
      Install   1 Package
      Upgrade  29 Packages
      Updates downloaded successfully.
    reason: HostUpgradeDone
    status: "True"
    type: HostUpgrades
  - lastHeartbeatTime: 2018-05-23T16:11:04Z
    lastTransitionTime: null
    reason: UpToDate
    status: "False"
    type: HostUpgradesReboot

SpComb commented 6 years ago

Testing with CentOS HostUpgradesReboot condition after a kernel upgrade:

  - lastHeartbeatTime: 2018-05-23T16:13:02Z
    lastTransitionTime: 2018-05-23T16:13:02Z
    reason: HostUpgradeDone
    status: "True"
    type: HostUpgrades
  - lastHeartbeatTime: 2018-05-23T16:13:02Z
    lastTransitionTime: 2018-05-23T16:13:02Z
    message: |
      Core libraries or services have been updated:
        kernel -> 3.10.0-862.3.2.el7

      Reboot is required to ensure that your system benefits from these updates.

      More information:
      https://access.redhat.com/solutions/27943
    reason: RebootRequired
    status: "True"
    type: HostUpgradesReboot

kontena / pharos-host-upgrades

Host upgrade status => kube node reboot condition #13