mkosi fixes

[lnussel]

Not sure how this worked before. Detecting mkosi recipes didn't work and some of the options used are deprecated. With the commits provided here, mkosi starts to build for me in pbuild but runs into issues later.

A config file for mkosi has to specify a distribution so mkosi knows eg what package manager to use. Then the distro comes with a set of default packages that are installed always. The build script cannot know about that before setting up the chroot as the list is part of a package of the target system.

cc @bluca

[bluca]

Mmh the current version works for me, last build was last week: https://build.opensuse.org/project/show/home:bluca:mkosi

Which version of mkosi did you try it with, and what's the config used?

[bluca]

I know some parameters changed, but I made sure we kept backward compat and accept them in the form used here, to avoid breaking things. That said, I don't test this with newer versions of mkosi, so it might have regressed (again), in which case I'll go and fix it (again)

[bluca]

A config file for mkosi has to specify a distribution so mkosi knows eg what package manager to use. Then the distro comes with a set of default packages that are installed always. The build script cannot know about that before setting up the chroot as the list is part of a package of the target system.

The way I set it up, it is simply detected from the build environment. IE, all builds are native builds - if you want a Debian testing image, you set up a build target that builds in Debian testing. The build script detects it from there.

Given the dependencies are resolved ahead of time, that seemed the most sensible thing to me. If you are building an image for distro X at version Y, it is most likely that in a build chroot of distro X at version Y you'll have all the building blocks needed.

[bluca]

Of course I'm fine with changing however needed to make it work - I'm rebuilding from latest main to see what @DaanDeMeyer broke this time, I should probably do the work to add the obs workflow integration to the github repo so that we get test coverage on PRs

[bluca]

I went and fought with the spec file&friends to build mkosi from main, and yeah it's currently not working:

https://build.opensuse.org/package/live_build_log/home:bluca:mkosi/fedora-image/Fedora_Rawhide_images/x86_64

[   24s] ‣ Syncing package manager metadata for default image
[   24s] bwrap: Can't find source path file:///.build.binaries/: No such file or directory
[   24s] bwrap: Can't find source path file:///.build.binaries/: No such file or directory
[   24s] ‣ "env HOME=/ dnf --assumeyes --best --releasever=rawhide --installroot=/var/tmp/mkosi-workspace_9a62ede/root --setopt=keepcache=1 --setopt=logdir=/var/log --setopt=cachedir=/var/cache/dnf --setopt=persistdir=/var/lib/dnf --setopt=install_weak_deps=0 --setopt=check_config_file_age=0 '--disableplugin=*' --enableplugin=builddep --nogpgcheck --setopt=metadata_expire=never --config=/etc/dnf/dnf.conf --setopt=reposdir=/etc/yum.repos.d --setopt=varsdir=/etc/dnf/vars makecache --refresh" returned non-zero exit code 1.

[bluca]

@lnussel with three fixes by Daan merged in main now builds work for me: https://build.opensuse.org/project/show/home:bluca:mkosi

One more thing that needs fixing is that the build-recipe-mkosi script gets stuck if there's no manifest, so we need to have [Output] ManifestFormat=json

[bluca]

Manifest fix that makes it optiona: https://github.com/openSUSE/obs-build/pull/986

[lnussel]

A config file for mkosi has to specify a distribution so mkosi knows eg what package manager to use. Then the distro comes with a set of default packages that are installed always. The build script cannot know about that before setting up the chroot as the list is part of a package of the target system.

The way I set it up, it is simply detected from the build environment. IE, all builds are native builds - if you want a Debian testing image, you set up a build target that builds in Debian testing. The build script detects it from there.

Given the dependencies are resolved ahead of time, that seemed the most sensible thing to me. If you are building an image for distro X at version Y, it is most likely that in a build chroot of distro X at version Y you'll have all the building blocks needed.

The mkosi file is evaluated like spec files outside of any distro context ie basically in empty space. So the OBS/pbuild/build only see what is in the mkosi file. Means the extra packages listed in /usr/lib/python3.11/site-packages/mkosi/resources/mkosi-tools/mkosi.conf.d/10-opensuse.conf and /usr/lib/python3.11/site-packages/mkosi/resources/mkosi-initrd/mkosi.conf.d/10-opensuse.conf are not known to the scheduler. Therefore those packages are not copied into /.build.binaries/ inside the build environment. mkosi is launched inside the build environment and does see the extra config files though. Therefore tries to make zypper install the extra packages and that fails. Short of keeping prjconf and those config files in sync all the time we'd need a parameter to mkosi to ignore any extra config files that were not specified on the command line I think.

[bluca]

Yeah those files are not used and shouldn't be used here, they are not useful. In my builds they are ignored as expected. I don't see that happening in my builds.

What configuration are you using?

This works fine for example:

https://build.opensuse.org/package/view_file/home:bluca:mkosi/fedora-image/mkosi.conf?expand=1

[lnussel]

Ok, was my own fault. Got it building in pbuild now.

[bluca]

Ok, was my own fault. Got it building in pbuild now.

Could you please share end-to-end instructions for pbuild? I've never managed to get mkosi to work with it, I test changes with an appliance VM which is more finicky and slower

[lnussel]

Ok, was my own fault. Got it building in pbuild now.

Could you please share end-to-end instructions for pbuild? I've never managed to get mkosi to work with it, I test changes with an appliance VM which is more finicky and slower

I used this: ~/git/obs-build/pbuild --dist tumbleweed --repo obs:/home:bluca:mkosi/openSUSE_Tumbleweed/x86_64 --repo https://download.opensuse.org/tumbleweed/repo/oss --repo https://download.opensuse.org/repositories/devel:/microos:/systemd-boot/openSUSE_Tumbleweed/ --single=suse-sysext --vm-disk-size=8192 --obs=https://api.opensuse.org/public/

[lnussel]

I merged 985 into this and got it working somehow but not fully. Building the sysext leads to hanging mkosi in the end. Looks like it picked up the base image at least. This patch is a dirty hack anyway. mkosi needs to follow the same path as kiwi basically and create a repo instead of using .build.binaries.

[bluca]

Thanks that's very helpful, will try and implement the obs-server side of things