kiwi-ng doesn't produce a tar.xz archive when building a docker package

[fluxcap1]

Problem description

For docker image builds, Kiwi-ng doesn't produce a tar.xz archive. Instead, I get image>.tar output.

I ran - sudo kiwi-ng --profile=docker system build --description ./ --target-dir images/

My config.xml

<image schemaversion="6.5" name="coldboot_container" xmlns:suse_label_helper="com.coldboot.label_helper">
  <description type="system">
    <author>coldboot</author>
    <contact>qm3h8f+l9assqdvm3no@grr.la</contact>
    <specification>openSUSE Tumbleweed container</specification>
  </description>
  <profiles>
    <profile name="docker" description="Docker-compatible image"/>
    <profile name="lxc" description="LXC-compatible image"/>
    <profile name="networkd" description="Image with systemd-networkd, suitable for use in systemd-machined"/>
  </profiles>
  <preferences profiles="docker">
    <type image="docker">
      <containerconfig
        name="coldboot/containerimages"
        tag="latest"
        maintainer="coldboot">
      </containerconfig>
    </type>
    <version>1.0.4</version>
    <packagemanager>zypper</packagemanager>
    <rpm-excludedocs>true</rpm-excludedocs>
  </preferences>
  <preferences profiles="lxc">
    <type image="tbz"/>
    <version>1.0.0</version>
    <packagemanager>zypper</packagemanager>
    <rpm-check-signatures>false</rpm-check-signatures>
    <rpm-excludedocs>true</rpm-excludedocs>
  </preferences>
  <preferences profiles="networkd">
    <type image="tbz"/>
    <version>1.0.0</version>
    <packagemanager>zypper</packagemanager>
    <rpm-check-signatures>false</rpm-check-signatures>
    <rpm-excludedocs>true</rpm-excludedocs>
  </preferences>

  <repository type="rpm-md" alias="Tumbleweed_OSS-updates" imageinclude="true" priority="97">
        <source path="http://download.opensuse.org/update/tumbleweed/"/>
  </repository>
  <repository type="yast2" alias="Tumbleweed_OSS" imageinclude="true" priority="98">
        <source path="https://download.opensuse.org/tumbleweed/repo/oss/"/>
  </repository>
  <repository type="yast2" alias="Tumbleweed_non-OSS" imageinclude="true" priority="98">
        <source path="https://download.opensuse.org/tumbleweed/repo/non-oss/"/>
  </repository>

  <packages type="image" profiles="networkd">
    <package name="systemd"/>
    <package name="systemd-network"/>
    <package name="systemd-sysvinit"/>
    <package name="iproute2"/>
  </packages>
  <packages type="image" profiles="lxc">
    <package name="systemd"/>
    <package name="systemd-sysvinit"/>
    <package name="wicked"/>
    <package name="iproute2"/>
  </packages>
  <packages type="bootstrap">
    <package name="aaa_base"/>
    <package name="bash"/>
    <package name="ca-certificates"/>
    <package name="ca-certificates-mozilla"/>
    <package name="coreutils"/>
    <package name="cracklib-dict-small"/>
    <package name="curl"/>
    <package name="filesystem"/>
    <package name="glibc-locale-base"/>
    <package name="shadow"/>
    <package name="util-linux"/>
    <package name="live-add-yast-repos"/>
    <package name="netcfg"/>
    <package name="openSUSE-build-key"/>
    <package name="openSUSE-release"/>
    <package name="patterns-base-fips"/>
    <package name="timezone"/>
    <package name="zypper"/>
  </packages>
</image>

My config.sh -

test -f /.profile && . /.profile
set -euxo pipefail
echo "Configure image: [$kiwi_iname]..."
suseImportBuildKey
add-yast-repos
zypper --non-interactive rm -u live-add-yast-repos
sed -i 's/.*solver.onlyRequires.*/solver.onlyRequires = true/g' /etc/zypp/zypp.conf
sed -i 's/.*rpm.install.excludedocs.*/rpm.install.excludedocs = yes/g' /etc/zypp/zypp.conf
shopt -s globstar
rm -f /usr/share/locale/**/*.mo
rm -f /var/lib/zypp/AnonymousUniqueId
rm -rf /var/log/{zypper.log,zypp/history,YaST2}
rm -rf /var/cache/zypp/*
arch=$(rpm -q --qf %{arch} glibc)
if [ "$arch" = "i586" ] || [ "$arch" = "i686" ]; then
        grep -q '^arch =' /etc/zypp/zypp.conf
fi
if [[ "$kiwi_profiles" == *"docker"* ]]; then
        rm /var/log/lastlog
fi
if [[ "$kiwi_profiles" == *"networkd"* ]]; then
        systemctl enable systemd-networkd
        systemctl enable systemd-resolved
        echo "L /etc/resolv.conf - - - - /run/systemd/resolve/stub-resolv.conf" > /etc/tmpfiles.d/stub-resolv.conf
fi
exit 0

Please help me with this.

OS and Software information

• KIWI version:
• Operating system host version: KIWI (next generation) version 9.24.49
• Operating system target version: openSUSE Tumbleweed VERSION="20221201"
• Open Build Service version (N/A if not using OBS):
• Koji version (N/A if not using Koji):

[Penguinbot4]

hello Interesting bld!!!

I have given up on kiwi for the moment the virt repo worked once then is shut down, I have the image where Python 3.10.8 and pip works. Trying to dup this with kiwi ,rpm and zypper/dnf is not happening tar is your friend

I want to install mamba and jyupterlab python 3.6 does not make this happen my build is leap15.5

please look here this would be my starting point zypp.conf zypper --non-interactive rm -u live-add-yast-repos sed -i 's/.solver.onlyRequires./solver.onlyRequires = true/g' /etc/zypp/zypp.conf sed -i 's/.rpm.install.excludedocs./rpm.install.excludedocs = yes/g' /etc/zypp/zypp.conf

[fluxcap1]

hello Interesting bld!!!

Thanks.

I want to install mamba and jyupterlab python 3.6 does not make this happen my build is leap15.5

If I find time, I will build that combination.

please look here this would be my starting point zypp.conf zypper --non-interactive rm -u live-add-yast-repos sed -i 's/.solver.onlyRequires./solver.onlyRequires = true/g' /etc/zypp/zypp.conf sed -i 's/.rpm.install.excludedocs./rpm.install.excludedocs = yes/g' /etc/zypp/zypp.conf

I don't think that is the issue here, it is meant for reducing the size of the image. I don't know why kiwi can't produce a tar.xz archive of the image.

[Penguinbot4]

im very sure you can add the compress task just where to put is the question my kiwi builds use image-root files that i can chroot into via leap15.5 arm64 read image=”tbz” A simple tar archive image. The tbz type packs the contents of the image root tree into a xz compressed tarball. https://osinside.github.io/kiwi/image_description/elements.html

[fluxcap1]

im very sure you can add the compress task just where to put is the question my kiwi builds use image-root files that i can chroot into via leap15.5 arm64 read image=”tbz” A simple tar archive image. The tbz type packs the contents of the image root tree into a xz compressed tarball. https://osinside.github.io/kiwi/image_description/elements.html

That is included in the tag in my config, and since that was not working, I triggered the issue.

[Penguinbot4]

yes i have this same issue Python 3.10.8 has no bz2 or sqlite3 and a few others

I have the debian sid code for the missing functions i have bzip2 installed i does not matter i need to compile against this and run configure again, this is great news pip installed Juypterlab no issue at all the take fail with no sqlite3 installed

I post the source and maybe we can issue a pull request this the python-3.10.8 build latest version you can see what is missing and my build line

my repos

[fluxcap1]

yes i have this same issue Python 3.10.8 has no bz2 or sqlite3 and a few others

Show your config files.

[Penguinbot4]

still in work ill post in the morning zypp is installing what it can what config files?

[fluxcap1]

still in work ill post in the morning zypp is installing what it can what config files?

kiwi files.

[Penguinbot4]

here they are i did not use Kiwi to build this release i did install the repo by using a ISO SUSE image 15,4 to create my chroot then RPM and tar to make a new image had many errors see my python 3.10.8 issue on this git repo

The kiwi images chroot would not install Python 3.10.8 due to compression errors. Pip would not install Python would this mean Juypterlab would not install and mambaforge has compress bz2 error also no install The missing Python modules are needed to run bz2 and to create xz files, Kiwi was built on Python 3.10.8 it will not build on 3.6 from Git clone this git repo

test-disk.tar.gz

[fluxcap1]

here they are i did not use Kiwi to build this release i did install the repo by using a ISO SUSE image 15,4 to create my chroot then RPM and tar to make a new image had many errors see my python 3.10.8 issue on this git repo

The kiwi images chroot would not install Python 3.10.8 due to compression errors. Pip would not install Python would this mean Juypterlab would not install and mambaforge has compress bz2 error also no install The missing Python modules are needed to run bz2 and to create xz files, Kiwi was built on Python 3.10.8 it will not build on 3.6 from Git clone this git repo

test-disk.tar.gz

Include python310-base to enable bz2 support. Anyway, I started this issue because of a problem I was having, but you aren't adding any useful information to it. On the SUSE forums, you should start a thread.

[Penguinbot4]

yes okay starting to get my dev env setup juypterlab is almost starting what SUSE forums let me know? xclients work now great still dbus is down and no machine ID yet LOL xfce4 work great

[Penguinbot4]

ok this the SUSE issue https://github.com/openSUSE/openSUSE-release-tools/issues/2905

[Penguinbot4]

hello i have bz2 and the open modules installed some are still missing

the config is show and the modules has a compile sequence I try to run you kiwi config

[Penguinbot4]

ok fluxcap image="tbz" for a docker image z@localhost:~/1> sh blddoc.sh [ INFO ]: 19:57:55 | Reading runtime config file: '/etc/kiwi.yml' [ INFO ]: 19:57:55 | Loading XML description [ INFO ]: 19:57:55 | Support for XML markup available [ INFO ]: 19:57:56 | Schematron validation failed: [ INFO ]: 19:57:56 | --> containerconfig(name) is only available for the following image types: docker oci appx [ ERROR ]: 19:57:56 | KiwiDescriptionInvalid: Failed to validate schema and/or schematron rules. Use --debug for more details

im building docker images via kiwi , your appl.kiwi has errors for syntax it seems appliance.kiwi.txt the attached file blds a docker image this is your file i tried to adapt appliance.kiwi.test.txt

i tried to run what you posted it fails with syntax errors have not had time to play with this

[schaefi]

In reply to the original question:

For docker image builds, Kiwi-ng doesn't produce a tar.xz archive. Instead, I get image>.tar output.

No compression is the default here. I think I remember it was done for a reason because the layers inside the tarball gets compressed by the umoci/skopeo or buildah tooling such that creating the archive compressed has not much benefit. @davidcassany please correct me when wrong.

In any case you can control if it should be compressed or not as follows:

Take a look at the file /etc/kiwi.yml it contains among others:

# Setup process parameters for container image creation
container:
  # Specify compression for container images
  # Possible values are true, false, xz or none.
  - compress: true

btw /etc/kiwi.yml is present if you have installed kiwi using a package, if you fetched from pip you don't get it because it's not part of the python module. If you don't have it on your system just look in the git here https://github.com/OSInside/kiwi/blob/master/kiwi.yml

Hope this helps

[fluxcap1]

In reply to the original question:

For docker image builds, Kiwi-ng doesn't produce a tar.xz archive. Instead, I get image>.tar output.

No compression is the default here. I think I remember it was done for a reason because the layers inside the tarball gets compressed by the umoci/skopeo or buildah tooling such that creating the archive compressed has not much benefit. @davidcassany please correct me when wrong.

In any case you can control if it should be compressed or not as follows:

Take a look at the file /etc/kiwi.yml it contains among others:

# Setup process parameters for container image creation
container:
  # Specify compression for container images
  # Possible values are true, false, xz or none.
  - compress: true

btw /etc/kiwi.yml is present if you have installed kiwi using a package, if you fetched from pip you don't get it because it's not part of the python module. If you don't have it on your system just look in the git here https://github.com/OSInside/kiwi/blob/master/kiwi.yml

Hope this helps

Please accept my apologies for the delayed response; I was confused after the previous commenter's nuisance posting.

I've these values in my kiwi.yml

bundle:
  - compress: true
  - has_package_changes: true
xz:
 - options: '--threads=4'
container:
  - compress: xz
oci:
  - archive_tool: umoci

RPMs are already compressed, so additional compression won't make much of a difference; I was curious as to why the images weren't being compressed even after selecting the compress option as instructed in the documentation. I'll attempt a smaller build tomorrow and report back.

Thank you.

[fluxcap1]

Another build I tried failed to produce tar.xz either.

[ INFO    ]: Processing: [########################################] 100%
[ INFO    ]: 10:10:57 | Importing Image description to system tree
[ INFO    ]: 10:10:57 | --> Importing state XML description to /home/flux/SOURCES/kiwi/Tangent_Container_Images/images/build/image-root/image/config.xml
[ INFO    ]: 10:10:57 | --> Importing config.sh script to /home/flux/SOURCES/kiwi/Tangent_Container_Images/images/build/image-root/image/config.sh
[ INFO    ]: 10:10:57 | --> Importing script helper functions
[ INFO    ]: 10:10:57 | Installing system (chroot) for build type: docker
[ INFO    ]: 10:10:57 | --> collection type: onlyRequired
[ INFO    ]: 10:10:57 | Check/Fix File Permissions
[ INFO    ]: 10:10:57 | Setting up image repository http://download.opensuse.org/update/tumbleweed/
[ INFO    ]: 10:10:57 | --> Type: rpm-md
[ INFO    ]: 10:10:57 | --> Translated: http://download.opensuse.org/update/tumbleweed/
[ INFO    ]: 10:10:57 | --> Alias: Tumbleweed_OSS-updates
[ INFO    ]: 10:10:57 | Setting up image repository https://download.opensuse.org/tumbleweed/repo/oss/
[ INFO    ]: 10:10:57 | --> Type: None
[ INFO    ]: 10:10:57 | --> Translated: https://download.opensuse.org/tumbleweed/repo/oss/
[ INFO    ]: 10:10:57 | --> Alias: Tumbleweed_OSS
[ INFO    ]: 10:10:57 | Setting up image repository https://download.opensuse.org/tumbleweed/repo/non-oss/
[ INFO    ]: 10:10:57 | --> Type: None
[ INFO    ]: 10:10:57 | --> Translated: https://download.opensuse.org/tumbleweed/repo/non-oss/
[ INFO    ]: 10:10:57 | --> Alias: Tumbleweed_non-OSS
[ INFO    ]: 10:10:57 | Calling config.sh script
[ INFO    ]: 10:11:14 | Using package manager backend: zypper
[ INFO    ]: 10:11:14 | Cleaning up SystemPrepare instance
[ INFO    ]: 10:11:14 | Creating system image
[ INFO    ]: 10:11:15 | Setting up docker container
[ INFO    ]: 10:11:15 | --> Creating container image
[ INFO    ]: 10:12:51 | Export rpm packages metadata
[ INFO    ]: 10:12:52 | Export rpm packages changelog metadata
[ INFO    ]: 10:12:52 | Export rpm verification metadata
[ INFO    ]: 10:12:55 | Result files:
[ INFO    ]: 10:12:55 | --> container: /home/flux/SOURCES/kiwi/Tangent_Container_Images/images/TangentOS_Container.x86_64-1.0.4.docker.tar
[ INFO    ]: 10:12:55 | --> image_changes: /home/flux/SOURCES/kiwi/Tangent_Container_Images/images/TangentOS_Container.x86_64-1.0.4.changes
[ INFO    ]: 10:12:55 | --> image_packages: /home/flux/SOURCES/kiwi/Tangent_Container_Images/images/TangentOS_Container.x86_64-1.0.4.packages
[ INFO    ]: 10:12:55 | --> image_verified: /home/flux/SOURCES/kiwi/Tangent_Container_Images/images/TangentOS_Container.x86_64-1.0.4.verified

[fluxcap1]

Attempted another smaller image but still no tar.xz

[ INFO    ]: Processing: [########################################] 100%
[ INFO    ]: 07:35:15 | Importing Image description to system tree
[ INFO    ]: 07:35:15 | --> Importing state XML description to /home/flux/SOURCES/kiwi/Tangent_Container_Images/TangentNginxContainer/images/build/image-root/image/config.xml
[ INFO    ]: 07:35:15 | --> Importing config.sh script to /home/flux/SOURCES/kiwi/Tangent_Container_Images/TangentNginxContainer/images/build/image-root/image/config.sh
[ INFO    ]: 07:35:15 | --> Importing script helper functions
[ INFO    ]: 07:35:15 | Installing system (chroot) for build type: docker
[ INFO    ]: 07:35:15 | --> collection type: onlyRequired
[ INFO    ]: 07:35:15 | Check/Fix File Permissions
[ INFO    ]: 07:35:15 | Setting up image repository http://download.opensuse.org/update/tumbleweed/
[ INFO    ]: 07:35:15 | --> Type: rpm-md
[ INFO    ]: 07:35:15 | --> Translated: http://download.opensuse.org/update/tumbleweed/
[ INFO    ]: 07:35:15 | --> Alias: Tumbleweed_OSS-updates
[ INFO    ]: 07:35:15 | Setting up image repository https://download.opensuse.org/tumbleweed/repo/oss/
[ INFO    ]: 07:35:15 | --> Type: None
[ INFO    ]: 07:35:15 | --> Translated: https://download.opensuse.org/tumbleweed/repo/oss/
[ INFO    ]: 07:35:15 | --> Alias: Tumbleweed_OSS
[ INFO    ]: 07:35:15 | Setting up image repository https://download.opensuse.org/tumbleweed/repo/non-oss/
[ INFO    ]: 07:35:15 | --> Type: None
[ INFO    ]: 07:35:15 | --> Translated: https://download.opensuse.org/tumbleweed/repo/non-oss/
[ INFO    ]: 07:35:15 | --> Alias: Tumbleweed_non-OSS
[ INFO    ]: 07:35:15 | Calling config.sh script
[ INFO    ]: 07:35:26 | Using package manager backend: zypper
[ INFO    ]: 07:35:26 | Cleaning up SystemPrepare instance
[ INFO    ]: 07:35:26 | Creating system image
[ INFO    ]: 07:35:26 | Setting up docker container
[ INFO    ]: 07:35:26 | --> Creating container image
[ INFO    ]: 07:36:52 | Export rpm packages metadata
[ INFO    ]: 07:36:52 | Export rpm packages changelog metadata
[ INFO    ]: 07:36:52 | Export rpm verification metadata
[ INFO    ]: 07:36:57 | Result files:
[ INFO    ]: 07:36:57 | --> container: /home/flux/SOURCES/kiwi/Tangent_Container_Images/TangentNginxContainer/images/nginx-container-image.x86_64-1.0.1_tn.docker.tar
[ INFO    ]: 07:36:57 | --> image_changes: /home/flux/SOURCES/kiwi/Tangent_Container_Images/TangentNginxContainer/images/nginx-container-image.x86_64-1.0.1_tn.changes
[ INFO    ]: 07:36:57 | --> image_packages: /home/flux/SOURCES/kiwi/Tangent_Container_Images/TangentNginxContainer/images/nginx-container-image.x86_64-1.0.1_tn.packages
[ INFO    ]: 07:36:57 | --> image_verified: /home/flux/SOURCES/kiwi/Tangent_Container_Images/TangentNginxContainer/images/nginx-container-image.x86_64-1.0.1_tn.verified

[schaefi]

ok thanks for the feedback. That smells like a bug. Sorry the inconvenience, I'll take a look and try to reproduce on my system.

[fluxcap1]

ok thanks for the feedback. That smells like a bug. Sorry the inconvenience, I'll take a look and try to reproduce on my system.

Okay, please take your time. Thank you.

[schaefi]

@fluxcap1 I think you really uncovered a problem in kiwi regarding the compression of containers. I did some tests because I knew we had added that options to control the compression. After some time I realized that the config setting will be taken into account, but only in the kiwi bundler. Maybe as a background information when you build an image with kiwi you can bundle the results for further processing after the build. So you call kiwi-ng system build ... and then you can call kiwi-ng result bundle ... At this result bundle call the compression setting would be handled and your container gets compressed

I think this needs to be changed because

1. It's misleading and not obvious from the config setting and the processing of the build
2. Users will not have an opportunity to create compressed containers without calling the kiwi result bundler
3. The default is uncompressed and that I will not change, but the assumption that the toolchain used to create the containers, umoci, skopeo, buildah will produce compressed results automatically is only correct under certain conditions. Thus having a way in kiwi to say; compress this docker, oci, appx, whatever archive during build makes a lot sense to me

Long story short. I will come up with a PR in a minute and I hope @davidcassany and others will have a look at it.

Thanks

[fluxcap1]

@fluxcap1 I think you really uncovered a problem in kiwi regarding the compression of containers. I did some tests because I knew we had added that options to control the compression. After some time I realized that the config setting will be taken into account, but only in the kiwi bundler. Maybe as a background information when you build an image with kiwi you can bundle the results for further processing after the build. So you call kiwi-ng system build ... and then you can call kiwi-ng result bundle ... At this result bundle call the compression setting would be handled and your container gets compressed

I think this needs to be changed because

1. It's misleading and not obvious from the config setting and the processing of the build

2. Users will not have an opportunity to create compressed containers without calling the kiwi result bundler

3. The default is uncompressed and that I will not change, but the assumption that the toolchain used to create the containers, umoci, skopeo, buildah will produce compressed results automatically is only correct under certain conditions. Thus having a way in kiwi to say; compress this docker, oci, appx, whatever archive during build makes a lot sense to me

Long story short. I will come up with a PR in a minute and I hope @davidcassany and others will have a look at it.

Thanks

Excellent, many thanks for the quick fix. Will the openSUSE package see this modification soon?

[penguingbot7]

so if you compress the tar file via xz post build does it make a difference ? Im not a docker user ,have used in the past just do not now

[schaefi]

so if you compress the tar file via xz post build does it make a difference ? Im not a docker user ,have used in the past just do not now

I depends on the data inside of the tar but yes it does make a big difference if the layers are not compressed. If you look at a kiwi build log for a container that uses skopeo to create the docker-archive it looks like this

[  387s] [ DEBUG   ]: 22:46:57 | EXEC: [skopeo copy oci:/var/tmp/kiwi_oci_dir.8wflb9_h/oci_layout:latest docker-archive:/usr/src/packages/KIWI-docker/kiwi-test-image-docker.x86_64-2.1.0.docker.tar:buildsystem:latest]

As kiwi creates containers from an OCI layout the first layer is just data. Given that data is a simple root tree of files the later compression has a huge impact. However, kiwi also supports creating containers from containers (derived_from) if that container has compressed layers and you just add a little bit more the compression of the result has not much effect. There are also other factors (encrypted data, other container formats e.g appx) which all impacts if the post compression has much benefit or not. That's why I think it makes sense to allow to configure this action.

[Penguinbot4]

yes okay on zypper or others package managers the files are decompressed on install with Jupyter-lab and mamabaforge the download the compressed file and unpack via pip python cmds my work is to try and find common ground for Debootstrap Kiwi and others the install repos are all different rpm, deb gentoo, arch, Suse. This is taking some time some use wget or curl other do not