search

kontena / pharos-host-upgrades

Kube DaemonSet for host OS upgrades
Apache License 2.0
41 stars 1 forks source link

scheduler: Prevent concurrent runs #5

Closed SpComb closed 6 years ago

SpComb commented 6 years ago

Fixes #3

Use a single goroutine loop triggered by an unbuffered chan to prevent concurrent scheduler runs.

2018/05/22 16:05:04 hosts/ubuntu probe success: systemd.HostInfo{KernelName:"Linux", Hostname:"ubuntu-xenial", OperatingSystemPrettyName:"Ubuntu 16.04.4 LTS", KernelVersion:"#153-Ubuntu SMP Sat May 19 10:58:46 UTC 2018", KernelRelease:"4.4.0-127-generic"}
2018/05/22 16:05:04 Probed host: hosts.HostInfo{OperatingSystem:"Ubuntu", OperatingSystemRelease:"16.04.4", Kernel:"Linux", KernelRelease:"4.4.0-127-generic"}
2018/05/22 16:05:04 Using --kube-namespace=kube-system --kube-daemonset=host-upgrades --kube-node=ubuntu-xenial
2018/05/22 16:05:04 kube/lock kube-system/daemonsets/host-upgrades: get
2018/05/22 16:05:04 kube/lock kube-system/daemonsets/host-upgrades: test pharos-host-upgrades.kontena.io/lock=: free
2018/05/22 16:05:04 Using kube lock kube-system/daemonsets/host-upgrades (acquired=false, value=)
2018/05/22 16:05:04 Using --schedule="@every 1s", first upgrade at: 2018-05-22 16:05:05 +0000 UTC m=+0.684922944 (in 576.410407ms)
2018/05/22 16:05:05 Acquiring kube lock...
2018/05/22 16:05:05 kube/lock kube-system/daemonsets/host-upgrades: wait
2018/05/22 16:05:05 kube/lock kube-system/daemonsets/host-upgrades: get
2018/05/22 16:05:05 kube/lock kube-system/daemonsets/host-upgrades: test pharos-host-upgrades.kontena.io/lock=: free
2018/05/22 16:05:05 kube/lock kube-system/daemonsets/host-upgrades: acquire
2018/05/22 16:05:05 kube/lock kube-system/daemonsets/host-upgrades: set pharos-host-upgrades.kontena.io/lock=ubuntu-xenial
2018/05/22 16:05:05 kube/lock kube-system/daemonsets/host-upgrades: update
2018/05/22 16:05:05 Running host upgrades...
2018/05/22 16:05:05 hosts/ubuntu upgrade: [/usr/bin/unattended-upgrade -v]
2018/05/22 16:05:05 systemd/exec host-upgrades.service: cmd=[/usr/bin/unattended-upgrade -v]
2018/05/22 16:05:05 systemd/exec host-upgrades.service: reset
2018/05/22 16:05:05 systemd/exec host-upgrades.service: start []dbus.Property{dbus.Property{Name:"ExecStart", Value:dbus.Variant{sig:dbus.Signature{str:"a(sasb)"}, value:[]dbus.execStart{dbus.execStart{Path:"/usr/bin/unattended-upgrade", Args:[]string{"/usr/bin/unattended-upgrade", "-v"}, UncleanIsFailure:false}}}}, dbus.Property{Name:"Type", Value:dbus.Variant{sig:dbus.Signature{str:"s"}, value:"oneshot"}}}
2018/05/22 16:05:05 systemd/exec host-upgrades.service: wait
2018/05/22 16:05:06 Scheduler is busy, skipping scheduled run
2018/05/22 16:05:07 Scheduler is busy, skipping scheduled run
2018/05/22 16:05:07 systemd/exec host-upgrades.service: journal 2018-05-22 16:05:05.367282 +0000 UTC: Initial blacklisted packages:
2018/05/22 16:05:07 systemd/exec host-upgrades.service: journal 2018-05-22 16:05:05.367491 +0000 UTC: Initial whitelisted packages:
2018/05/22 16:05:07 systemd/exec host-upgrades.service: journal 2018-05-22 16:05:05.367834 +0000 UTC: Starting unattended upgrades script
2018/05/22 16:05:07 systemd/exec host-upgrades.service: journal 2018-05-22 16:05:05.368319 +0000 UTC: Allowed origins are: ['o=Ubuntu,a=xenial', 'o=Ubuntu,a=xenial-security', 'o=UbuntuESM,a=xenial']
2018/05/22 16:05:07 systemd/exec host-upgrades.service: journal 2018-05-22 16:05:07.374615 +0000 UTC: No packages found that can be upgraded unattended and no pending auto-removals
2018/05/22 16:05:07 systemd/exec host-upgrades.service: done
2018/05/22 16:05:07 kube/lock kube-system/daemonsets/host-upgrades: get
2018/05/22 16:05:07 kube/lock kube-system/daemonsets/host-upgrades: release
2018/05/22 16:05:07 kube/lock kube-system/daemonsets/host-upgrades: clear pharos-host-upgrades.kontena.io/lock=ubuntu-xenial
2018/05/22 16:05:07 kube/lock kube-system/daemonsets/host-upgrades: update
2018/05/22 16:05:07 Schedule run completed in 2.422517226s, next upgrade at: 2018-05-22 16:05:08 +0000 UTC m=+3.684922925 (in 573.56086ms)
2018/05/22 16:05:08 Acquiring kube lock...
2018/05/22 16:05:08 kube/lock kube-system/daemonsets/host-upgrades: wait
2018/05/22 16:05:08 kube/lock kube-system/daemonsets/host-upgrades: get
2018/05/22 16:05:08 kube/lock kube-system/daemonsets/host-upgrades: test pharos-host-upgrades.kontena.io/lock=: free
2018/05/22 16:05:08 kube/lock kube-system/daemonsets/host-upgrades: acquire
2018/05/22 16:05:08 kube/lock kube-system/daemonsets/host-upgrades: set pharos-host-upgrades.kontena.io/lock=ubuntu-xenial
2018/05/22 16:05:08 kube/lock kube-system/daemonsets/host-upgrades: update
2018/05/22 16:05:08 Running host upgrades...
2018/05/22 16:05:08 hosts/ubuntu upgrade: [/usr/bin/unattended-upgrade -v]
2018/05/22 16:05:08 systemd/exec host-upgrades.service: cmd=[/usr/bin/unattended-upgrade -v]
2018/05/22 16:05:08 systemd/exec host-upgrades.service: reset
2018/05/22 16:05:08 systemd/exec host-upgrades.service: start []dbus.Property{dbus.Property{Name:"ExecStart", Value:dbus.Variant{sig:dbus.Signature{str:"a(sasb)"}, value:[]dbus.execStart{dbus.execStart{Path:"/usr/bin/unattended-upgrade", Args:[]string{"/usr/bin/unattended-upgrade", "-v"}, UncleanIsFailure:false}}}}, dbus.Property{Name:"Type", Value:dbus.Variant{sig:dbus.Signature{str:"s"}, value:"oneshot"}}}
2018/05/22 16:05:08 systemd/exec host-upgrades.service: wait
2018/05/22 16:05:09 Scheduler is busy, skipping scheduled run