search

kairos-io / kairos

The immutable Linux meta-distribution for edge Kubernetes.
https://kairos.io
Apache License 2.0
1.16k stars 96 forks source link

Auroraboot out of memory for some images #3037

Open jimmykarily opened 19 hours ago

jimmykarily commented 19 hours ago

Discussed in https://github.com/kairos-io/kairos/discussions/2986

Originally posted by **davidnajar** November 7, 2024 Hello Community! I'm having issues when building some large images using Auroraboot. I'm customizing the Rocky OS image by creating a new container adding extra layers on top on it, using helm and skopeo to download oci images to the local filesystem with the intention to later on, after installing, and during first booting, using a systemd service, reading this images with skopeo again and push it to the local containerd daemon. This works good in airgap enviroments, to prevent the need of internet connection to set up my edge cluster's default services. However, I've been adding extra docker images now, and I've arrived to a point where I'm getting always out of memory errors when trying to build the iso using auroraboot. The docker image itself uncompressed is about 5.4 gb according to docker desktop. The last available iso file i was able to build has been around 2.4 gb . But now, after adding a couple more images, I'm not able to build anymore. While no documented, when reading the source code of auroraboot (and not a go expert) I got to the conclusion that i can limit the size of the memory pressure by setting a value in system.memory option. I run this build in a CI agent: ``` docker run -i --rm --net=host --cap-add=NET_ADMIN \ --mount 'type=bind,source=/var/run/docker.sock,target=/var/run/docker.sock' \ --mount 'type=bind,source={{.HOST_VOLUME}}/build/container,target=/build' \ --workdir /build \ quay.io/kairos/auroraboot:\ --debug=true \ --set "disable_http_server=true" \ --set "disable_netboot=true" \ --set "state_dir=/build" \ --set "disk.vhd=false" \ --set "system.memory=2500" \ # settings I've been playing with --set "system.cores=2" \ # settings I've been playing with --set "container_image=docker://previous-custom-rockylinux-docker-image:latest" \ --cloud-config - < src/cloud-init/cloud-config.yaml ``` When doing this, what i see at the end is a failure showing the following: ``` Pulling container image 'previous-custom-rockylinux-docker-image:latest' to '/build/temp-rootfs') fatal error: runtime: out of memory ``` I understand that the error is that might be the image is too big to be loaded in memory but I can't confirm. However, I was expecting that by setting system.memory to some value to be able to limit that. Is there some setting I could be missing?