search

kontena / pharos-host-upgrades

Kube DaemonSet for host OS upgrades
Apache License 2.0
41 stars 1 forks source link

Volume mount for /run/log/journal fails if persistent /var/log/journal is setup #15

Open SpComb opened 6 years ago

SpComb commented 6 years ago

The transient /run/log/journal directory will not exist if the persistent /var/log/journal directory has been setup, and this causes the volume mounts for the pod to fail. I don't know if it's possible to mount both journal paths for the daemonset without interfering with the systemd-journald operation by creating the /var/log/journal directory as a side-effect. This makes pod/container access to the host journal problematic, given the lack of a dbus API for journald... This might make the use of systemd for execs a bad idea, since it relies on journal access for the output logging functionality.

  Warning  FailedMount            3m (x8 over 18m)    kubelet, ubuntu-xenial  Unable to mount volumes for pod "host-upgrades-8pgtq_kube-system(ade33b5f-600b-11e8-a164-02a3be24b14f)": timeout expired waiting for volumes to attach or mount for pod "kube-system"/"host-upgrades-8pgtq". list of unmounted volumes=[journal]. list of unattached volumes=[config host dbus journal host-upgrades-token-9bn6d]
  Warning  FailedMount            26s (x18 over 20m)  kubelet, ubuntu-xenial  MountVolume.SetUp failed for volume "journal" : hostPath type check failed: /run/log/journal is not a directory
$ ls -ld /var/log/journal/ /run/log/journal
ls: cannot access '/run/log/journal': No such file or directory
drwxr-sr-x+ 3 root systemd-journal 4096 May 25 11:10 /var/log/journal/
SpComb commented 6 years ago

Somewhat mitigated by the /*/log/journal being optional... if neither is mounted to the container, then the systemd execs will simply not read anything from the journal.

I suppose we could just omit the journal mounts from the daemonset, but it would mean that you need to manually journalctl -u host-upgrades on the host to debug exec errors...