c9s: rpm-ostreed fails to start when installing an OKD cluster

[karmab]

rpm-ostreed fails to start (which causes machine config operator to fail too) Error shows as

A dependency job for rpm-ostreed.service failed. See 'journalctl -xe' for details.
○ rpm-ostreed.service - rpm-ostree System Management Daemon
     Loaded: loaded (/usr/lib/systemd/system/rpm-ostreed.service; static)
    Drop-In: /etc/systemd/system/rpm-ostreed.service.d
             └─10-mco-default-env.conf
             /run/systemd/system/rpm-ostreed.service.d
             └─bug2111817.conf
             /etc/systemd/system/rpm-ostreed.service.d
             └─mco-controlplane-nice.conf
     Active: inactive (dead)
       Docs: man:rpm-ostree(1)

Jul 20 13:03:48 okd-ctlplane-0.karmalabs.corp systemd[1]: Dependency failed for rpm-ostree System Management Daemon.
Jul 20 13:03:48 okd-ctlplane-0.karmalabs.corp systemd[1]: rpm-ostreed.service: Job rpm-ostreed.service/start failed with result 'dependency'.
Jul 20 13:08:59 okd-ctlplane-0.karmalabs.corp systemd[1]: Dependency failed for rpm-ostree System Management Daemon.
Jul 20 13:08:59 okd-ctlplane-0.karmalabs.corp systemd[1]: rpm-ostreed.service: Job rpm-ostreed.service/start failed with result 'dependency'.
Jul 20 13:14:00 okd-ctlplane-0.karmalabs.corp systemd[1]: Dependency failed for rpm-ostree System Management Daemon.
Jul 20 13:14:00 okd-ctlplane-0.karmalabs.corp systemd[1]: rpm-ostreed.service: Job rpm-ostreed.service/start failed with result 'dependency'.
Jul 20 13:19:07 okd-ctlplane-0.karmalabs.corp systemd[1]: Dependency failed for rpm-ostree System Management Daemon.
Jul 20 13:19:07 okd-ctlplane-0.karmalabs.corp systemd[1]: rpm-ostreed.service: Job rpm-ostreed.service/start failed with result 'dependency'.
Jul 20 13:21:59 okd-ctlplane-0.karmalabs.corp systemd[1]: Dependency failed for rpm-ostree System Management Daemon.
Jul 20 13:21:59 okd-ctlplane-0.karmalabs.corp systemd[1]: rpm-ostreed.service: Job rpm-ostreed.service/start failed with result 'dependency'.
error: Loading sysroot: exit status: 1

When trying to manually start the daemon, we get

[root@okd-ctlplane-0 ~]# rpm-ostree start-daemon
error: Couldn't start daemon: Error setting up sysroot: loading sysroot: Unexpected state: /run/ostree-booted found, but no /boot/loader directory

[karmab]

An other error that might be related is with this service

systemd-fsck@dev-disk-by\x2duuid-edbe9990\x2d28f5\x2d453e\x2db044\x2d1b2b601d785a.service

which fails because

/dev/vda3 has unsupported feature(s): FEATURE_C12

fix for this seems to be to bring a more recent version of e2fsck

[jlebon]

That's an artifact of pivoting from Fedora to c9s. It should be fixed by using c9s bootimages once we get past all the SELinux issues. See also https://github.com/coreos/coreos-assembler/commit/1c280e775cd9bd13bc2356cdc980b742eab3c321.

[karmab]

Is there an estimated date for when this will happen/be fixed?

[jlebon]

This should be tracked on the OKD side instead.