Custom kairos-debian ISO with systemd-journald errors on boot

[jimmykarily]

Discussed in https://github.com/kairos-io/kairos/discussions/1614

^{Originally posted by **massimogallina** July 18, 2023} I am using a custom version of kairos-debian ISO to boot a VirtualBox VM. The custom ISO is built in this way: ``` docker run -v ./build:/tmp/auroraboot \ -v ./config.yaml:/config.yaml \ --rm quay.io/kairos/auroraboot \ --set container_image=${REGISTRY}/${REPOSITORY}:${CI_COMMIT_TAG} \ --set "disable_http_server=true" \ --set "disable_netboot=true" \ --set "state_dir=/tmp/auroraboot" \ --cloud-config /config.yaml ``` The container image is generated with this Dockerfile: ``` FROM quay.io/kairos/kairos-debian:v2.3.0-k3sv1.27.3-k3s1 # Add all additional packages to install on kairos RUN sudo apt-get update && sudo apt-get install -y curl wireguard unzip iputils-ping easy-rsa tcpdump cron vim # Install AWS CLI RUN sudo curl "https://awscli.amazonaws.com/awscli-exe-linux-x86_64.zip" -o "awscliv2.zip" && sudo unzip awscliv2.zip && sudo ./aws/install -b /usr/bin -i /opt/awscli && rm awscliv2.zip # Set version RUN touch /etc/edgegaldi_version RUN echo "edgegaldi-os:" > /etc/edgegaldi_version ``` and the `config.yaml` file is the following one: ``` #cloud-config install: auto: true device: "auto" reboot: false poweroff: true users: - name: "test" passwd: "test123" ssh_authorized_keys: - "ssh-rsa ..........." groups: - "admin" shell: /bin/bash homedir: "/home/test" ``` When I boot the VM, after the grub boot the following messages start to be printed continuosly and the VM is not able to complete the boot: ``` Data hash table of /run/log/journal/####/system.journal has a fill level at 75.0 (3273 of 4359 items, 8388608 file size, 627 bytes per hash table item), suggesting rotation /run/log/journal/####/system.journal: Journal header limits reached or header out-of-date, rotating ``` If I use the same process but using the kairos image `quay.io/kairos/kairos-debian:v2.1.0-k3sv1.26.4-k3s1` in the Dockerfile, the VM is able to start without printing any logs about sytemd-journald. How can I solve this issue? Thanks, M

[Itxaka]

I could reproduce this and was not able to boot from livecd.

adding rd.live.overlay.overlayfs to the grub livecd entry made it pas and install went correctly and everything come up correctly after that:

test@localhost:~$ cat /etc/edgegaldi_version 
edgegaldi-os:<TAG>
test@localhost:~$ aws 

usage: aws [options] <command> <subcommand> [<subcommand> ...] [parameters]
To see help text, you can run:

  aws help
  aws <command> help
  aws <command> <subcommand> help

aws: error: the following arguments are required: command

test@localhost:~$ which aws
/usr/bin/aws
test@localhost:~$ ls /opt/awscli/v2/
2.13.1  current
test@localhost:~$ cat /etc/os-release 
PRETTY_NAME="Debian GNU/Linux trixie/sid"
NAME="Debian GNU/Linux"
VERSION_CODENAME=trixie
ID=debian
HOME_URL="https://www.debian.org/"
SUPPORT_URL="https://www.debian.org/support"
BUG_REPORT_URL="https://bugs.debian.org/"
KAIROS_NAME="kairos-debian"
KAIROS_VERSION="v2.3.0-k3sv1.27.3+k3s1"
KAIROS_ID="kairos"
KAIROS_ID_LIKE="kairos-debian"
KAIROS_VERSION_ID="v2.3.0-k3sv1.27.3+k3s1"
KAIROS_PRETTY_NAME="kairos-debian v2.3.0-k3sv1.27.3+k3s1"
KAIROS_BUG_REPORT_URL="https://github.com/kairos-io/kairos/issues/new/choose"
KAIROS_HOME_URL="https://github.com/kairos-io/provider-kairos"
KAIROS_IMAGE_REPO="quay.io/kairos/kairos-debian"
KAIROS_IMAGE_LABEL="latest"
KAIROS_GITHUB_REPO="kairos-io/provider-kairos"
KAIROS_VARIANT="kairos"

[massimogallina]

Hi @Itxaka, so your suggestion is to edit the grub during the ISO generation and add the rd.live.overlay.overlayfs entry , for example following this guide: https://kairos.io/docs/reference/auroraboot/#override-grub-config-file ?