Fedora 32 is not being included in ISO when using `qubes-builder`

Qubes OS version

R4.0

Affected component(s) or functionality

qubes-builder

Brief summary

Fedora 32 template is not being included in the iso built using qubes-builder, and if that was the only template specified as DISTS_VM then the built image has no template to create any VM.

How Reproducible

100%

To Reproduce

Steps to reproduce the behavior:

Follow the steps in https://www.qubes-os.org/doc/qubes-iso-building/
Specify fc32 or fc32 buster as DISTS_VM
Notice how the ISO doesn't include fedora 32

Expected behavior

Fedora 32 should be included in the ISO, or in the worst case, the build should error out if no template was included in the built ISO.

Actual behavior

Fedora 32 is being excluded from the ISO

Screenshots

qubes-r4 1

Additional context

These are the changes made to builder.conf:

diff --git a/example-configs/qubes-os-master.conf b/builder.conf
index 78478a4..855a056 100644
--- a/example-configs/qubes-os-master.conf
+++ b/builder.conf
@@ -4,12 +4,13 @@

 GIT_BASEURL ?= https://github.com
 GIT_PREFIX ?= QubesOS/qubes-
-NO_SIGN ?= 1
+GIT_CLONE_FAST=1
+SIGN_KEY=redacted

 BACKEND_VMM=xen

-DIST_DOM0 ?= fc32
-DISTS_VM ?= fc32 fc33 buster
+DIST_DOM0=fc32
+DISTS_VM=fc32

 MGMT_COMPONENTS = \
        mgmt-salt \

Solutions you've tried

include buster and then install fedora 32 template using qubes-dom0-update

Happening to me too (when building 4.1).

rm -rf work && mkdir work
rm -rf /tmp/dnfroot
# Update installer repo
/usr/bin/createrepo_c -q -g ../../conf/comps-dom0.xml --update yum/qubes-dom0
# Destination directory for RPM
mkdir -p /home/user/qubes-src/installer-qubes-os/work/20210415/x86_64/os/Packages
# Copy Fedora key to DNF installroot
mkdir -p /tmp/dnfroot/etc/pki/rpm-gpg
cp /home/user/qubes-src/installer-qubes-os/qubes-release/RPM-GPG-KEY-fedora-32-primary /tmp/dnfroot/etc/pki/rpm-gpg
# Provide qubes-release/conf for Qubes keys
mkdir -p /tmp/dnfroot/tmp/qubes-installer
ln -nsf /home/user/qubes-src/installer-qubes-os /tmp/dnfroot/tmp/qubes-installer
# Legacy conf using this folder
ln -nsf /home/user/qubes-src/installer-qubes-os /tmp/qubes-installer
# Extract repos conf and packages from kickstart
mkdir -p /tmp/dnfroot/etc/yum.repos.d
/home/user/qubes-src/installer-qubes-os/scripts/ksparser --ks /home/user/qubes-src/installer-qubes-os/conf/qubes-kick
start.cfg --extract-repo-conf-to /tmp/dnfroot/etc/yum.repos.d/installer.repo --extract-packages-to /tmp/dnfroot/tmp/p
ackages.list
umask 022; /usr/bin/dnf -y --releasever=32 --installroot=/tmp/dnfroot --downloaddir=/home/user/qubes-src/installer-qu
bes-os/work/20210415/x86_64/os/Packages --downloadonly install @core @base @base-x @xfce-desktop-qubes @xfce-extra-pl
ugins @xfce-media @sound-basic @fonts @hardware-support @qubes @qubes-ui @anaconda-tools @fedora @debian @whonix --ex
clude=adobe-source-code-pro-fonts --exclude=compat-f32-dejavu-sans-fonts --exclude=compat-f32-dejavu-sans-mono-fonts 
--exclude=compat-f32-dejavu-serif-fonts --exclude=crypto-policies-scripts --exclude=deltarpm --exclude=dnfdaemon-seli
nux --exclude=fips-mode-setup --exclude=flac --exclude=fwupd-plugin-flashrom --exclude=fwupd-plugin-modem-manager --e
xclude=gcc-gdb-plugin --exclude=geoclue2 --exclude=geolite2-city --exclude=geolite2-country --exclude=gnupg2-smime --
exclude=gstreamer1-plugins-good-qt --exclude=lame --exclude=libsss_autofs --exclude=libsss_sudo --exclude=libxcrypt-c
ompat --exclude=libyui-gtk --exclude=libyui-mga-gtk --exclude=libyui-mga-qt --exclude=libyui-qt --exclude=libyui-qt-g
raph --exclude=mkpasswd --exclude=ntfs-3g-system-compression --exclude=oddjob-mkhomedir --exclude=openssl-pkcs11 --ex
clude=opus-tools --exclude=perl-IO-Compress --exclude=perl-IO-Socket-SSL --exclude=perl-Math-BigInt --exclude=perl-Mo
zilla-CA --exclude=pigz --exclude=pinentry --exclude=python-systemd-doc --exclude=python-unversioned-command --exclud
e=python3-unbound --exclude=rpm-plugin-systemd-inhibit --exclude=sssd-nfs-idmap --exclude=trousers --exclude=xorg-x11
-fonts-misc --exclude=xdg-desktop-portal-gtk --exclude=blueberry --exclude=gnome-bluetooth --exclude=gnome-bluetooth-
libs
fedora                                          7.7 MB/s |  70 MB     00:09    
fedora-updates                                  3.7 MB/s |  29 MB     00:08    
installer                                       538  B/s | 257  B     00:00    
qubes-dom0                                      1.6 MB/s | 420 kB     00:00    
dom0-updates                                    2.4 kB/s | 257  B     00:00    
No match for group package "qubes-template-fedora-33"
No match for group package "tabish-eeyek-fonts"
No match for group package "qubes-template-whonix-gw-15"
No match for group package "qubes-template-debian-10"
No match for group package "authconfig"
No match for group package "qubes-template-whonix-ws-15"
Dependencies resolved.

Perhaps it's pulling from https://github.com/QubesOS/qubes-meta-packages/blob/f63409d2fc74c23a9577baef95cbd8bd1868783b/comps/comps-dom0.xml#L1440 ?

Have you built the template? If not, it would never be included because it's simply not here...

Perhaps it's pulling from https://github.com/QubesOS/qubes-meta-packages/blob/f63409d2fc74c23a9577baef95cbd8bd1868783b/comps/comps-dom0.xml#L1440 ?

Yes and if you look at what's not here (No match for group package "qubes-template-fedora-33"), Fedora 32 is not the default Fedora template anymore but 33.

Perhaps it's pulling from https://github.com/QubesOS/qubes-meta-packages/blob/f63409d2fc74c23a9577baef95cbd8bd1868783b/comps/comps-dom0.xml#L1440 ?

Yes and if you look at what's not here (No match for group package "qubes-template-fedora-33"), Fedora 32 is not the default Fedora template anymore but 33.

Ah alright, perhaps we want to update the options in ./setup to add fedora 32 and 33 options (the latest option there was fedora31)? https://github.com/QubesOS/qubes-builder/pull/167

Perhaps it's pulling from https://github.com/QubesOS/qubes-meta-packages /blob/f63409d2fc74c23a9577baef95cbd8bd1868783b/comps/comps-dom0.xml#L1440 ?

Thanks for the pointer, I've managed to accomplish what I wanted adjusting some files there. It's worth noting that I needed to change the files in qubes-src/installer-qubes-os instead of qubes-src/meta-packages, as the later was not having effect in the resulting iso. Here are the changes I needed to do in order to make a build with "extra" templates:

Apply in qubes-src/installer-qubes-os

diff --git a/conf/qubes-kickstart.cfg b/conf/qubes-kickstart.cfg
index 9626457..9962785 100644
--- a/conf/qubes-kickstart.cfg
+++ b/conf/qubes-kickstart.cfg
@@ -23,9 +23,11 @@ repo --name=dom0-updates --baseurl=file:///tmp/qubes-installer/yum/dom0-updates/
 @qubes
 @qubes-ui
 @anaconda-tools
-@fedora
+@fedora33
+@fedora32
 @debian
 @whonix
+@archlinux
 # weaks dependencies
 -adobe-source-code-pro-fonts
 -compat-f32-dejavu-sans-fonts

Apply in qubes-src/installer-qubes-os/meta-packages

diff --git a/comps/comps-dom0.xml b/comps/comps-dom0.xml
index 4f38c33..14067d2 100644
--- a/comps/comps-dom0.xml
+++ b/comps/comps-dom0.xml
@@ -1431,11 +1431,21 @@
     </packagelist>
   </group>
   <group>
-    <id>fedora</id>
+    <id>fedora32</id>
+    <name>Fedora 32 template</name>
+    <description>Fedora 32 template</description>
+    <default>false</default>
+    <uservisible>true</uservisible>
+    <packagelist>
+      <packagereq>qubes-template-fedora-32</packagereq>
+    </packagelist>
+  </group>
+  <group>
+    <id>fedora33</id>
     <name>Fedora 33 template</name>
     <description>Fedora 33 template</description>
     <default>false</default>
-    <uservisible>false</uservisible>
+    <uservisible>true</uservisible>
     <packagelist>
       <packagereq>qubes-template-fedora-33</packagereq>
     </packagelist>
@@ -1445,7 +1455,7 @@
     <name>Debian 10 (buster) template</name>
     <description>Debian 10 (buster) template</description>
     <default>false</default>
-    <uservisible>false</uservisible>
+    <uservisible>true</uservisible>
     <packagelist>
       <packagereq>qubes-template-debian-10</packagereq>
     </packagelist>
@@ -1461,6 +1471,16 @@
       <packagereq>qubes-template-whonix-ws-15</packagereq>
     </packagelist>
   </group>
+  <group>
+    <id>archlinux</id>
+    <name>Archlinux</name>
+    <description>Archlinux template</description>
+    <default>false</default>
+    <uservisible>true</uservisible>
+    <packagelist>
+      <packagereq>qubes-template-archlinux</packagereq>
+    </packagelist>
+  </group>
   <environment>
    <id>qubes-xfce</id>
    <display_order>3</display_order>

And it resulted in this:

install

There's a few couple notes:

You can't select the checkboxes in the 1st phase for (fedora, debian, whonix) or they will error out in 2nd phase "Initial Setup" saying that they're already installed (hence why the default is uservisible=false)
For any other vm that is not listed in the 2nd phase, one needs to run: rpm --reinstall /var/lib/qubes/vm-templates/$VM_NAME.rpm for lvm or qvm-template-postprocess post-install --really $VM_NAME /var/lib/qubes/vm-templates/$VM_NAME for btrfs in order to finish the installation process and for them to show in qubes tools.
It would be great if all the templates included were listed in the 1st phase, so users don't have to go through the installation process to know what templates the iso includes (like the screenshot above, but without checkboxes for those that shouldn't be checked in that step)

@fepitre Is this the way of adding extra templates to the resulting iso or there's a better way? I've checked a lot of the documentation & reading the repository sources but didn't find anything there.

This issue is being closed because:

This issue is on the "Release 4.0 updates" milestone.
Qubes OS 4.0 reached EOL (end-of-life) over one year ago.
There has not been any activity on this issue in over one year.

If anyone believes that this issue should be reopened and reassigned to an active milestone, please leave a brief comment. (For example, if a bug still affects Qubes OS 4.1, then the comment "Affects 4.1" will suffice.)

QubesOS / qubes-issues

Fedora 32 is not being included in ISO when using `qubes-builder` #6529