Cannot build image with custom partition.

Hi,

We are trying to build ubuntu image and have specific requirements for creating custom disk partitions.

Problem description

While trying to build the image (oem) with custom partition , I get the error KiwiFileSystemSyncError ( see the snippet below)

config.xml snippet:

<type image="oem" filesystem="ext4" initrd_system="dracut" installiso="true" firmware="uefi" installboot="install" bootpartition="true" bootpartsize="1024" efipartsize="200">
      <bootloader name="grub2"></bootloader>
      <oemconfig>
        <oem-swap>true</oem-swap>
        <oem-swapsize>4096</oem-swapsize>
        <oem-resize>false</oem-resize>
        <oem-unattended>true</oem-unattended>
        <oem-device-filter>/dev/ram</oem-device-filter>
        <oem-multipath-scan>false</oem-multipath-scan>
     </oemconfig>
     <partitions>
        <partition name="bootefi" size="1G" mountpoint="/boot/efi" filesystem="ext4" />
        <partition name="altroot" size="10G" mountpoint="/altroot" filesystem="ext4"/>
     </partitions>

Error snippet: 2024-06-13T18:19:55.4343398Z [ INFO ]: 18:19:55 | Syncing system to image 2024-06-13T18:19:55.4344151Z [ INFO ]: 18:19:55 | --> Syncing custom partition(s) data 2024-06-13T18:19:55.4346280Z [ ERROR ]: 18:19:55 | KiwiFileSystemSyncError: given root directory /tmp/output/build/image-root/altroot/ does not exist

Expected behaviour

We should be able to build image with custom partitions.

Steps to reproduce the behavior

Build image with custom partition.

OS and Software information

KIWI version: 9.23.20 and on the latest.
Operating system host version: ubuntu 22.04 LTS
Operating system target version: ubuntu 22.04 LTS
Open Build Service version (N/A if not using OBS): https://download.opensuse.org/repositories/Virtualization:/Appliances:/Builder/openSUSE_Leap_15.5/ kiwi-appliance-builder
Koji version (N/A if not using Koji):

Hi, yeah two issues here:

/boot/efi belongs to the bootloader setup and the EFI boot standard which requires the EFI partition to be a FAT partition. Setting this as custom partition and to ext4 could cause further issues probably not at build time but at boot time. Do you have a particular reason to inject partitions to the bootloader space ? It should actually not be needed.
The other custom partition which also caused the error you saw is related to the mountpoint="/altroot" this is not an FHS compliant mount point and therefore does by default not exist in your system. We intentionally don't automatically create missing mountpoints because we want this to be explicitly stated in the image description. Thus to fix this please add something like the following in your config.sh script

# create altroot mount point for reason XYZ
mkdir -p /altroot

This will at least fix the mountpoint issue

@schaefi : Thanks I will check and redo this. Meanwhile, I wanted to check if it is possible to :

Create root partition something like : partition name="rootfs" size="10G" mountpoint="/" filesystem="ext4"
And then other volumes lvm on other partition : systemdisk name="testgroup" preferlvm="true" -volume mountpoint="/var" name="var" size="5G" -volume mountpoint="/images" name="images" size="48G" Please advice .

ok, not sure if I got it all right. From the above you can specify a type like this

<type image="oem" filesystem="ext4" firmware="efi" ... >
    <systemdisk name="vgsystem">
        <volume name="@root=rootfs" size="10G"/>
        <volume name="var" size="5G"/>
    </systemdisk>
    <partitions>
        <partition name="images" size="48G" mountpoint="/images" filesystem="ext4"/>
    </partitions>
</type>

Create a LVM partition and place one volume group named vgsystem and place two volumes for "rootfs(/) 10G" and "/var 5G" into it, setup as ext4
Create a Linux partition 48G, setup as ext4, mounted to /images

I think this is not completely matching what you want, as it seems you want /var and /images to live on another partition and in another volume group. To do this you need a bit more because kiwi doesn't support multiple volume groups in the declarative description. So to get there you can do the following

<type image="oem" filesystem="ext4" firmware="efi" editbootconfig="custom_vg_create.sh" editbootinstall="custom_vg_setup.sh" ... >
    <systemdisk>
        <volume name="@root=rootfs" size="10G"/>
    </systemdisk>
    <partitions>
        <partition name="extensions" size="60G">
    </partitions>
</type>

This produces the rootfs in one LVM as you want it and creates an empty partition (60G) for the other layout. The other layout needs to be created by your own code though via the custom_vg_create.sh and the custom_vg_setup.sh script hooks. They can look like the following:

custom_vg_create.sh

image_fs=$1
root_partnum=$2
root_device=/dev/loop*p${root_partnum}
loop_name=$(basename $root_device | cut -f 1-2 -d'p')
extension_device=/dev/${loop_name}p3

pvcreate ${extension_device}
vgcreate testgroup ${extension_device}
lvcreate -Zn -L 5G -n var testgroup
lvcreate -Zn -L 48G -n images testgroup

custom_vg_setup.sh

image_file=$1
root_device=$2
loop_name=$(basename $root_device | cut -f 1-2 -d'p')
disk_device=/dev/${loop_name}

# mount root part
root=$(mktemp -d /tmp/rootmount-XXX)
mount /dev/${loop_name}p4 $root || exit 1

# move /var to extra part and volume
part=$(mktemp -d /tmp/partmount-XXX)
mount /dev/testgroup/var $part && mv $root/var/* $part/
umount --lazy $part && rmdir $part

# update fstab
mkdir -p $root/images
echo "/dev/testgroup/var /var ext4 defaults 0 0" >> $root/etc/fstab
echo "/dev/testgroup/images /images ext4 defaults 0 0" >> $root/etc/fstab

# umount root part
umount --lazy $root && rmdir $root

# cleanup maps
partx --delete $disk_device

As you can see this is a bit more effort and personally I think it's not a good idea to move a system partition (/var) outside of the main OS volumegroup and/or partition. If you are using LVM you can add physical extends to the volume group as you like, also from different partitions or new block devices.

I assume you have good reasons for this layout and I hope this information helps to move you forward

I've had to implement something similar, in order to get "CIS Hardened Images" out of the box using KiwiNG. Here is my xml config:

    <preferences>
        <type image="oem" filesystem="ext4" kernelcmdline="console=ttyS0 console=tty1" firmware="uefi" format="vmdk" bootpartition="true" bootpartsize="3000">
            <bootloader name="grub2" console="console" timeout="5"/>
            <systemdisk name="vgr">
                <volume name="var_vol" size="120G" mountpoint="var"/>
                <volume name="var_tmp_vol" size="4G" mountpoint="/var/tmp/"/>
                <volume name="var_log_vol" size="6G" mountpoint="/var/log/"/>
                <volume name="var_log_audit_vol" size="4G" mountpoint="/var/log/audit"/>
                <volume name="home_vol" size="120G" mountpoint="home"/>
                <volume name="tmp_vol" size="4G" mountpoint="tmp"/>
                <volume name="@root=root_vol" freespace="40G"/>
            </systemdisk>
            <oemconfig>
                <oem-resize>false</oem-resize>
                <!-- <oem-swap>true</oem-swap> -->
                <!-- <oem-swapsize>2048</oem-swapsize> -->
                <!-- <oem-swapname>swap_vol</oem-swapname> -->
            </oemconfig>
        </type>
    </preferences>

So I've given /boot 3G of space, because I am lazy, and I don't want to clear the old kernels very often, but you do you :smile:

I also use swapfile, instead of a swap partition to be more flexible (that's why it's commented out).

Thanks much for you help. I am sorry I could not check this quick, since I was occupied with something else.

I did check this as suggested by : @schaefi by creating custom_vg_create.sh and custom_vg_setup.sh .

But I get the error as below , not sure what is the issue : KiwiCommandError: mount: stderr: mount: /var/tmp/kiwi_mount_manager.87p7_ovv: unknown filesystem type 'LVM2_member'

Log snippet here:

        _2024-06-27T16:15:56.5802581Z [ INFO    ]: 16:15:56 | Syncing system to image
        2024-06-27T16:15:56.5803543Z [ INFO    ]: 16:15:56 | --> Syncing custom partition(s) data
        2024-06-27T16:15:56.5807451Z [2;37m[ DEBUG   ]: 16:15:56 | EXEC: [mountpoint -q /var/tmp/kiwi_mount_manager.o1ai39yk]
        2024-06-27T16:15:56.5846518Z [2;37m[ DEBUG   ]: 16:15:56 | EXEC: [mount /dev/loop4p4 /var/tmp/kiwi_mount_manager.o1ai39yk]
        2024-06-27T16:15:56.5915165Z [2;37m[ DEBUG   ]: 16:15:56 | Check for extended attributes on /var/tmp/kiwi_mount_manager.o1ai39yk said: [Errno 61] No data available: '/var/tmp/kiwi_mount_manager.o1ai39yk'
        2024-06-27T16:15:56.5918348Z [2;37m[ DEBUG   ]: 16:15:56 | EXEC: [rsync --archive --hard-links --xattrs --acls --one-file-system --inplace /tmp/output/build/image-root/altroot/ /var/tmp/kiwi_mount_manager.o1ai39yk]
        2024-06-27T16:15:56.6376330Z [ INFO    ]: 16:15:56 | --> Syncing EFI boot data to EFI partition
        2024-06-27T16:15:56.6380243Z [2;37m[ DEBUG   ]: 16:15:56 | EXEC: [mountpoint -q /var/tmp/kiwi_mount_manager.bj502544]
        2024-06-27T16:15:56.6417661Z [2;37m[ DEBUG   ]: 16:15:56 | EXEC: [mount /dev/loop4p2 /var/tmp/kiwi_mount_manager.bj502544]
        2024-06-27T16:15:56.6478547Z [2;37m[ DEBUG   ]: 16:15:56 | Check for extended attributes on /var/tmp/kiwi_mount_manager.bj502544 said: [Errno 95] Operation not supported: '/var/tmp/kiwi_mount_manager.bj502544'
        2024-06-27T16:15:56.6480662Z [3;33m[ WARNING ]: 16:15:56 | Extended attributes not supported for target: /var/tmp/kiwi_mount_manager.bj502544
        2024-06-27T16:15:56.6482319Z [2;37m[ DEBUG   ]: 16:15:56 | EXEC: [rsync --archive --hard-links --one-file-system --inplace /tmp/output/build/image-root/boot/efi/ /var/tmp/kiwi_mount_manager.bj502544]
        2024-06-27T16:15:56.7107036Z [ INFO    ]: 16:15:56 | --> Syncing boot data at extra partition
        2024-06-27T16:15:56.7110102Z [2;37m[ DEBUG   ]: 16:15:56 | EXEC: [mountpoint -q /var/tmp/kiwi_mount_manager.87p7_ovv]
        2024-06-27T16:15:56.7147338Z [2;37m[ DEBUG   ]: 16:15:56 | EXEC: [mount /dev/loop4p3 /var/tmp/kiwi_mount_manager.87p7_ovv]
        2024-06-27T16:15:56.7204563Z [2;37m[ DEBUG   ]: 16:15:56 | **EXEC: Failed with stderr: mount: /var/tmp/kiwi_mount_manager.87p7_ovv: unknown filesystem type 'LVM2_member'.**
        2024-06-27T16:15:56.7206441Z [3;31m[ ERROR   ]: 16:15:56 | **KiwiCommandError: mount: stderr: mount: /var/tmp/kiwi_mount_manager.87p7_ovv: unknown filesystem type 'LVM2_member'.**


My config.xml be like this :

         <type image="oem" filesystem="ext4" initrd_system="dracut" installiso="true" firmware="uefi" installboot="install" efipartsize="1024" bootfilesystem="ext4" editbootconfig="custom_vg_create.sh" editbootinstall="custom_vg_setup.sh" >
                  <bootloader name="grub2"/>
                   <partitions>
                          <partition name="altroot" filesystem="ext4" size="10G" mountpoint="/altroot"/>
                   </partitions>
                  <oemconfig>
                        <oem-resize>false</oem-resize>
                 </oemconfig>
                 <systemdisk>
                       <volume name="@root" label="ROOTFS" size="10G"/>
                </systemdisk>
         </type>



custom_vg_create.sh and custom_vg_setup.sh are exactly same as you suggested earlier.

OSInside / kiwi